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FOREWORD 


The  primary  goal  of  the  21st  Century  Army,  Force  XXI,  is  to  optimize  capabilities 
through  information  technology.  Organization  centered  on  weapon  systems  will  give 
way  to  organization  centered  on  information  exchange  that  leads  to  unified  action.  Force 
XXI  will  dominate,  control  and  win  through  an  information-based  battle  command 
structure  that  functions  with  greater  adaptability,  fluidity  and  decisiveness  than  has  ever 
been  seen  before.  The  Army  is  being  redesigned  for  the  information  age  by  defining  its 
doctrine,  training,  leaders,  organization,  materiel  and  soldiers  according  to  knowledge 
gained  from  experimentation  carried  out  through  the  Battle  Lab  process.  Advanced 
Technology  Demonstrations  and  Advanced  Warfighting  Experiments. 

Alterations  made  to  introduce  new  technologies  that  provide  increased  generation 
and  transmission  of  data  will  not,  alone,  enable  the  Army  to  reach  its  goal.  Data  are  not 
information,  information  is  not  knowledge,  knowledge  is  not  wisdom.  Information 
technology,  in  the  form  of  hardware  and  software  that  generate,  manage  and  display 
information  for  a  command  staff  may  provide  an  orderly  and  rapid  presentation  of 
relevant  information  and  therefore  provide  the  command  staff  more  time  to  deliberate  its 
course  of  action.  It  can  not  provide  that  staff  with  ready  made  judgment  or  qualitative 
evaluation.  The  hardware  and  software  can  not  think  for  them.  To  reach  its  goal  the 
Army  must  devise  a  command  structure  that  considers  the  staff  and  its  digital  decision 
aids  as  an  integrated  system. 

This  report  provides  an  analysis  of  the  behavior  of  a  command  staff  in  a  digitized, 
simulated  operational  environment  and  presents  recommendations  for  the  design  of  those 
decision  aids  and  the  training  that  will  be  required  to  achieve  efficient  operation  of  that 
integrated  command  system.  The  report  also  provides  recommendations  for  the 
methodologies  needed  to  perform  the  behavioral  research  necessary  to  continue  the 
redesign  process.  The  results  of  this  work  are  relevant  to  policy,  doctrine  and  training 
program  decisions. 

Results  and  findings  from  this  research  were  briefed  to  COL  William  Powell, 
Director  of  Training  Doctrine  and  Simulation,  U.S.  Army  Aviation  Center,  COL  Gary 
Coleman,  Deputy  Director,  Air  Maneuver  Battle  Lab  and  COL  Russell  Forshag,  Director, 
Warfighting  Futures  Division,  Directorate  of  Training  Doctrine  and  Simulation,  on 
March  4,  1998.  Results  and  findings  were  briefed  to  LTG  William  Bolt,  Deputy  Chief  of 
Staff  for  Training,  and  to  MG  Daniel  Petrosky,  Commander,  U.S.  Aviation  Center  and 
Fort  Rucker,  on  March  5,  1998.  Results  and  findings  were  briefed  to  LTG  Montgomery 
Meigs,  Commander,  Combined  Arms  Center,  on  March  24, 1998. 
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Technical  Director 


USE  AND  MANAGEMENT  OF  DIGITAL  INFORMATION  BY  ARMY 
AVIATION  BATTALION  BATTLE  STAFF  MEMBERS 

EXECUTIVE  SUMMARY 


Research  Requirement: 

This  report  results  from  a  set  of  requirements  submitted  to  the  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  1998  Research-Based 
Personnel  and  Training  Study  and  Analysis  Program  by  the  Directorate  of  Training, 
Doctrine,  and  Simulation,  U.S.  Army  Aviation  Center  at  Fort  Rucker,  Alabama.  This 
work  has  been  carried  out  as  part  of  Work  Package  2151:  Force  XXI  Aviation  Battle 
Staff  Training:  Integration  of  C4I  Into  Battle  Simulation  and  has  been  produced  through 
the  cooperative  efforts  of  the  Army  Research  Institute  and  the  U.S.  Army  Aviation 
Center. 


The  requirements  statement  listed  five  requirements,  four  of  which  are  addressed 
in  the  research  reported  here:  1.  develop  baseline  tactics,  techniques,  and  procedures  for 
utilizing  digital  Command,  Control,  Communications,  Computers  and  Intelligence 
systems  to  perform  the  military  decision  making  process,  2.  develop  baseline  tactics, 
techniques  and  procedures  for  monitoring  execution  of  the  decision  support  template 
utilizing  digital  systems,  3.  develop  a  recommended  display  configuration  which  best 
supports  the  commander’s  information  requirements  and  4.  determine  whether  an 
information  manager  is  required  in  the  digital  tactical  operations  center  and  who  that 
person  should  be. 

Procedure: 

This  effort  is  based  on  empirical,  nonexperimental  research  embedded  within  a 
training  simulation  exercise  that  constituted  the  capstone  event  of  staff  officer  training  for 
the  Aviation  branch.  Time  and  resources  were  insufficient  to  permit  an  experimental 
approach  nor  would  experimentation  be  efficient  given  the  objectives  that  resulted  from 
the  requirements. 

The  study  was  conducted  in  the  Army  Aviation  Test  Bed  and  the  Aviation 
Warfighting  Simulation  Center,  interconnected  within  a  fiber  optic  cable  network.  Both 
facilities  are  reconfigurable  simulation  centers  that  support  a  wide  range  of  Army 
aviation  research.  Brigade  and  battalion  tactical  operations  centers  were  replicated  in 
these  centers  with  partial  implementations  of  the  Army  Tactical  Command  and  Control 
System  digitized  suite.  The  brigade  installation  served  as  a  control  center.  Observational 
data  were  compiled  from  events  and  activities  in  the  battalion  installation.  Each  tactical 
operations  center  had  three  cells:  a  current  operations  cell,  a  plans  cell,  plus  an 
administrative  and  logistics  operation  center. 
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Over  a  five-day  period  a  student  battalion  battle  staff  conducted  mission  planning 
and  execution  functions  within  an  operational  scenario  presented  in  a  virtual  simulation 
that  was  implemented  by  a  confederation  of  computer  based  operational  and  tactical 
simulation  models.  Battalion  operations  were  conducted  in  response  to  orders  issued  by 
the  simulated  brigade  staff.  These  components  simulated  both  friendly  and  enemy 
maneuver  forces.  A  ten  member  observer  staff  recorded  events  during  mission  planning, 
execution  and  during  after  action  reviews  according  to  formats  established  to  address 
issues  relevant  to  the  requirements  and  objectives.  The  student  battle  staff  members 
completed  questionnaires  covering  their  background  and  experience  prior  to  the  exercise, 
their  activities  and  experiences  during  mission  planning,  and  a  questionnaire  pertaining  to 
their  impressions  relating  to  training  potential  effectiveness.  Five  key  staff  members 
were  interviewed  during  the  week  following  the  exercise. 

Findings: 

Despite  the  constraints  in  the  design  of  the  exercise,  this  study  yielded  useful 
information  about  problems  that  can  be  solved  by  developing  new  tactics,  techniques  and 
procedures  and  the  general  descriptive  information  concerning  them.  Problems  with 
planning  time  and  distribution  of  planning  tasks  among  members  of  the  battle  staff 
indicate  a  need  for  more  formalized  workload  distribution  and  time  management  under 
both  normal  and  time  constrained  conditions  with  an  emphasis  on  tailoring  planning 
activities  to  available  time.  There  is  also  a  need  to  specify  planning  activities  that  can  be 
carried  out  concurrently.  An  implication  of  this  is  a  need  for  increased  cross  training 
among  the  incumbents  of  staff  positions. 

There  is  a  need  to  specify  the  manner  in  which  each  digital  subsystem  of  the 
digitized  suite  should  be  used  to  accomplish  each  mission-planning  step.  Emphasis 
should  be  placed  on  the  unique  capabilities  of  each  subsystem  and  efficient  transfer  of 
data  among  them.  There  is  also  a  need  for  improved  management  of  graphic  information, 
particularly  the  origins  and  ages  of  icons  and  control  measures.  This  study  provided  no 
evidence  that  adding  an  information  manager  to  the  battle  staff  is  the  best  way  to  improve 
the  management  of  information.  This  problem  would  be  better  addressed  by  governing 
the  information  management  activities  and  responsibilities  of  existing  battle  staff 
members.  Specific  areas  of  opportunity  include  distribution  of  information  monitoring 
activities,  screening  incoming  information,  and  anticipation  of  information  needs 

This  study  yielded  no  specific  information  about  the  problems  that  battle  staff 
members  encounter  when  monitoring  the  execution  of  a  decision  support  template.  Other 
information  gained  from  the  study  can  be  used  to  address  general  requirements  to 
remediate  problems  with  the  decision  support  template  that  are  likely  to  be  encountered. 
Battle  staff  procedures  should  minimize  the  requirements  for  battle  staff  members  to 
remember  the  location  of  decision  points  and  trigger  events,  possibly  through  special 
graphic  overlay  that  depicts  the  template.  The  responsibility  for  monitoring  the  execution 
of  decision  support  templates  should  be  shared  by  two  or  more  battle  staff  members.  The 
workload  for  this  should  be  integrated  into  their  other  tasks  to  prevent  overload. 
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In  addition,  there  appears  to  be  a  need  to  ensure  that  all  information  from  prior  missions 
is  removed  from  the  digital  systems,  ensure  that  all  echelons  employ  a  common  file- 
naming  date-time  label  protocols,  and  ensure  that  all  files  transferred  among  systems  or 
echelons  are  accompanied  by  a  free  text  message  describing  their  contents. 

There  is  a  need  to  govern  the  use  and  implementation  of  pull-down  menus  to 
prevent  obscuration  of  map  information  while  a  menu  is  in  use.  In  addition,  battle  staff 
members  need  a  more  direct  method  for  scrolling  within  a  map  display  and  changing  the 
level  of  map  magnification.  There  were  many  instances  in  which  battle  staff  members 
were  unable  to  discriminate  important  information  that  was  portrayed  on  the  large-format 
wall  mounted  display.  Inspection  revealed  that  alphanumeric  and  point  symbols  were 
depicted  with  too  few  raster  lines,  inadequate  height  or  inadequate  figure-ground 
contrast.  Control  measure  symbology  often  obscured  map  detail.  Three  approaches  to 
reducing  the  severity  of  obscuration  are  suggested.  One  is  simply  to  reduce  the  area 
covered  by  symbols.  The  amount  that  symbol  size  can  be  reduced  and  still  be  easily 
legible  depends  on  the  factors  that  influence  display  resolution.  A  second  approach  is  to 
color  symbols  rather  than  shape  code  them.  A  third  approach  is  to  provide  a  capability 
to  temporarily  remove  symbols  that  obscure  important  map  detail.  Ideally,  the  digital 
systems  would  be  programmed  to  enable  operators  to  temporarily  remove  a  single 
symbol,  all  symbols  of  a  specified  class,  or  all  symbolic  information  depicted. 

There  is  a  need  to  coordinate  the  level  of  map  detail  with  the  displayed  map 
magnification.  This  implies  software  that  selects  classes  of  topographic  features  and 
changes  the  rules  of  topographic  feature  generalization  depending  on  the  size  of  the  area 
portrayed.  Alternatively,  key  battle  staff  members  might  be  provided  with  duplicate 
displays  except  for  topographic  database  scale.  This  solution  may  increase  the 
requirements  for  equipment,  space,  and  personnel,  but  is  technically  feasible  at  the 
present  time. 

Regimental  icons  commonly  used  to  depict  the  locations  of  enemy  forces  are  not 
adequate  in  some  situations.  Display  systems  should  enable  battle  staff  members  to 
deaggregate  the  icons  to  the  company  level.  The  effectiveness  of  tracking  and  display  of 
friendly  aircraft  locations  would  be  improved  with  identification  tags  attached  to  the  their 
display  icons.  There  is  a  need  to  modify  threat  icons  to  depict  the  age  of  the  intelligence 
information  underlying  it.  The  color,  shape,  or  brightness  of  a  threat  icon  could  be 
changed  systematically  to  reflect  its  age.  Likewise,  threat  icons  should  indicate  their 
condition  as  active  or  destroyed. 

The  study  also  yielded  lessons  about  the  methodology.  In  future  studies  of  this 
type,  every  attempt  should  be  made  to  acquire  the  time  and  resources  needed  to  provide 
battle  staff  members  with  comprehensive  training  on  the  functions  and  operation  of  the 
digital  systems.  Instructional  utility  may  be  improved  by  increasing  the  volume  of 
intelligence  information  that  must  be  processed  during  mission  planning  and  during 
mission  execution.  The  comparatively  low  volume  of  intelligence  traffic  in  this  exercise 
caused  the  workload  of  some  battle  staff  members  to  be  unrealistically  low. 
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In  future  studies  of  this  type,  consideration  should  be  given  to  face-to-face 
interviews  with  individual  battle  staff  members  as  a  means  of  obtaining  detailed 
information  about  mission  planning  problems  and  oversights.  Some  mission  planning 
problems  and  oversights  do  not  become  apparent  until  after  the  mission  has  been 
executed.  Interviews  to  identify  specific  mission  planning  problems  and  oversights  are 
likely  to  be  most  effective  if  they  are  conducted  immediately  after  the  mission  has  been 
executed. 

A  nonexperimental  study  may  not  be  the  most  effective  way  to  define  tactics, 
techniques  and  procedures.  Even  with  a  fully  proficient  battle  staff  this  study  could  have 
yielded  information  only  about  the  tactics,  techniques  and  procedures  that  were  used. 
Optimization  of  tactics,  techniques  and  procedures  will  more  likely  result  from  an 
experimental  approach  in  which  performance  is  measured  as  the  alternatives  are  varied 
systematically. 

The  present  study  indicates  that  the  amount  of  time  required  to  perform  the 
planning  steps,  individually  or  collectively,  is  not  a  reliable  measure  of  the  proficiency  of 
the  battle  staff  members  who  perform  them.  Perhaps  the  most  important  factor  is  the 
tendency  of  a  battle  staff  to  use  all  of  the  planning  time  that  is  available.  Another  factor 
is  that  the  characteristics  of  the  mission  have  an  enormous  influence  in  the  amount  of 
time  that  is  required  to  plan  the  mission. 

Additional  training  of  observers  is  needed  to  improve  their  ability  to  recognize 
key  events  and  problems  that  occur  during  mission  execution.  For  example,  additional 
training  probably  would  increase  observers’  ability  to  recognize  and  record  key  decision 
points  and  instances  of  heavy  workload.  However,  it  seems  unlikely  that  additional 
training  would  significantly  improve  observers’  ability  to  recognize  decision  errors, 
problems  encountered  in  operating  the  digital  equipment,  and  problems  encountered  in 
interpreting  the  digital  system  displays. 

Passive  observation  by  relatively  inexperienced  observers  can  yield  a  great  deal  of 
useful  information  about  activities  and  problems  that  occur  during  mission  execution. 
However,  there  are  ways  to  improve  the  information  that  is  recorded  by  the  observers. 

One  way  is  to  replace  the  voice  communication  system  used  in  this  study  with  a  system 
in  which  headphones  rather  than  speakers  are  used.  With  speakers,  it  is  impossible  for 
observers  to  track  the  flow  of  information  to  and  from  each  battle  staff  member.  That  is, 
it  is  impossible  for  observers  to  determine  the  battle  staff  members  who  hear  messages 
that  are  announced  over  the  speakers. 

Analysis  of  voice  communication  events  yields  valuable  information  about 
information  management  problems,  the  absolute  and  relative  workload  of  battle  staff 
members,  and  command  style.  Although  very  labor  intensive,  the  recording  and  analysis 
of  communication  events  is  highly  recommended  for  future  research  on  the  use  of  digital 
command  and  control  systems.  Development  of  automated  systems  to  facilitate  this  by 
reducing  labor  and  processing  time  is  a  recommended.  Automation  of  communication 
event  recording  and  analysis  could  also  provide  enhanced  training. 
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Utilization  of  Findings: 

These  findings  are  provided  to  the  U.S.  Army  Aviation  Center  and  other  Training 
and  Doctrine  Command  centers  to  facilitate  policy,  doctrine  and  training  program 
decisions.  They  pertain  to  the  design  and  development  of  improved  digitized  battle  staff 
equipment  and  organization,  establishment  of  training  requirements  for  duty  in  brigade 
and  battalion  staff  positions,  design  and  development  of  staff  training  programs  and  to  the 
design  of  future  research  involving  group  interactions  in  medium  and  large  scale  human- 
computer  systems. 
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USE  AND  MANAGEMENT  OF  DIGITAL  INFORMATION  BY 
ARMY  AVIATION  BATTALION  BATTLE  STAFF  MEMBERS 

Introduction 

Background 


Requirements 

The  impetus  for  this  study  was  a  set  of  requirements  issued  by  the  Directorate  of 
Training,  Doctrine,  and  Simulation  (DOTDS),  U.S.  Army  Aviation  Center  (USAAVNC),  Fort 
Rucker,  AL.  A  DOTDS  requirements  statement,  dated  30  January  1997,  listed  the  five 
requirements  shown  below. 

•  Develop  a  plan  for  the  integration  of  digital  Command,  Control,  Communications, 
Computers  and  Intelligence  (C4I)  systems  into  aviation  leader  training. 

•  Develop  baseline  tactics,  techniques,  and  procedures  (TTPs)  for  utilizing  digital  C4I 
systems  to  perform  the  military  decision  making  process  (MDMP). 

•  Develop  baseline  TTPs  for  monitoring  execution  of  the  decision  support  template 
(DST)  utilizing  digital  C4I  systems. 

•  Develop  a  recommended  C4I  system  display  configuration  which  best  supports  the 
commander’s  information  requirements. 

•  Determine  whether  an  information  manager  is  required  in  the  digital  tactical 
operations  center  (TOC)  and  who  that  person  should  be. 

These  requirements  were  submitted  to  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  (ARI)  1998  Research-Based  Personnel  and  Training  Study  and 
Analysis  Program  as  a  result  of  the  USAAVNC  response  to  a  request  from  the  Training  and 
Doctrine  Command  (TRADOC)  for  input  to  the  Office  of  the  Deputy  Chief  of  Staff  for  Training 
(ODCST)  study  requirements.  The  objective  of  the  Study  and  Analysis  Program  is  to  conduct 
rapid  turn-around  investigations  in  support  of  Army  policy  and  doctrine.  Studies  conducted 
within  this  program  typically  use  behavioral  and  social  science  knowledge  and  expertise  plus 
existing  data  bases  to  provide  information  for  policy  and  doctrine  decisions.  The  request 
submitted  by  the  USAAVNC  was  selected  and  entered  into  the  program  as  Work  Package  2151: 
Force  XXI  Aviation  Battle  Staff  Training:  Integration  of  C4I  into  Battle  Simulation.  The  work 
reported  here  results  from  a  cooperative  effort  among  ARI,  USAAVNC  DOTDS,  the  1st  Aviation 
Brigade,  the  Directorate  of  Combat  Developments  (DCD)  and  the  Air  Maneuver  Battle  Lab 
(AMBL)  as  principle  parties. 

The  efforts  reported  here  were  designed  to  address  all  of  the  above  requirements  except 
the  first.  The  first  requirement  will  be  met  through  nonempirical  methods  (e.g.,  review  of 
relevant  literature,  interviews  with  subject  matter  experts  (SMEs)  and  reported  in  a  separate 
document. 
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The  Army  Battle  Command  System 


The  cornerstone  of  the  Army’s  plan  to  meet  the  challenges  of  the  21st  Century  is  the 
development  of  the  capability  to  achieve  information  dominance  on  the  battlefield.  The 
achievement  of  information  dominance  requires  the  means  to  (a)  destroy,  disrupt,  and  control  the 
enemy’s  capability  to  compile  and  distribute  information  and  (b)  provide  our  commanders  with 
accurate,  relevant  information  quickly  enough  for  them  to  use  it  effectively.  The  Army  has  a 
plan  for  development  and  introduction  of  digitized  systems  into  the  command  structure  to  help 
achieve  this  condition. 

ABCS.  This  plan  organizes  digitization  efforts  around  the  force  structure  and  levels  of 
command.  Digitally  equipped  TOCs  are  included  for  echelons  above  corps  (EAC)  and  at  corps, 
division,  brigade  and  battalion  levels  with  additional  digitization  capabilities  applied  below  the 
battalion  level.  At  each  level  a  suite  of  computer  based  automated  applications  operates  within  a 
local  area  net  interconnected  to  other  nets  at  levels  above,  below  and  parallel  within  the 
command  structure.  At  the  apex  is  the  Army  Battle  Command  System  (ABCS)  which  comprises 
the  Global  Command  and  Control  System- Army  (GCCS-A)  for  strategic,  theater  and  EAC 
applications,  the  Army  Tactical  Command  and  Control  System  (ATCCS),  which  applies  to  corps 
through  battalion  levels,  and  the  Force  XXI  Battle  Command  Brigade  and  Below  (FBCB2) 
system  which  represents  the  evolution  of  a  system  known  as  Applique. 

The  GCCS-A  is  intended  to  facilitate  force  tracking,  support  for  conducting  civilian 
affairs  and  host  nation  coordination,  theater  air  defense,  targeting,  psychological  operations, 
command  and  control,  logistics  plus  medical  and  personnel  status  support.  It  provides  a  standard 
format  and  method  for  worldwide  theater  command  and  control,  combat  service  support  for 
EAC,  facilities  for  planning  and  execution  of  mobilization,  deployment  and  sustainment,  and 
supports  joint,  combined,  and  allied  operations  in  addition  to  special  operations  and  operations 
other  than  war. 

The  ATCCS  comprises  five  main  subsystems  plus  four  supporting  subsystems.  The  five 
main  subsystems  are  the  Maneuver  Control  System  (MCS),  All  Source  Analysis  System 
(ASAS),  Advanced  Field  Artillery  Tactical  Data  System  (AFATDS),  Air  and  Missile  Defense 
Workstation  (AMDW/S)  and  Combat  Service  Support  Control  System  (CSSCS).  The  four 
supporting  subsystems  are  the  Integrated  Meteorological  System  (IMETS),  Digital 
Topographical  Support  System  (DTSS),  Common  Ground  Station  (CGS)  and  Battlefield 
Planning  /  Visualization  (BPV)  system. 

The  FBCB2  is  intended  primarily  for  installation  on  individual  vehicles  although  it  may 
be  found  in  TOCs.  It  provides  integrated  battle  command  capability  to  commanders  at  all  levels 
from  brigade  down.  It  integrates  internal  capabilities  of  weapon  and  sensor  systems  for 
generating  and  processing  information  with  parallel  systems  and  information  nodes  above  and 
below.  FBCB2  provides  near  real  time  display  of  friendly  unit  locations,  reported  enemy  and 
unidentified  unit  locations,  spot  reports,  calls  for  fire,  environmental  hazard  reports  (nuclear  and 
biological  contamination)  and  medical  evacuation  requests.  FBCB2  also  facilitates  creation  and 
dissemination  of  operational  orders  and  tactical  overlays  plus  logistics  and  personnel  reports. 
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The  ABCS  is  designed  around  the  requirement  for  interoperability  among  nodes  and 
levels.  This  is  accomplished  primarily  through  a  suite  of  communications  devices  with  a  high 
degree  of  commonality  throughout  the  system.  These  devices  include  Mobile  Subscriber 
Equipment  (MSE),  generally  present  at  brigade  and  above  command  nodes,  an  Integrated 
Network  Controller  (INC)  which  manages  message  traffic  through  routing  functions,  the 
Enhanced  Position  Location  Reporting  System  (EPLRS),  a  digital  radio  system  with  integrated 
automatic  position  information,  complex  security  measures  and  traffic  conflict  resolution,  the 
Single  Channel  Ground  Air  Radio  System  (SINCGARS),  which  handles  voice  and  digital 
communication  in  a  prioritized  mode,  the  Near  Term  Digital  Radio  (NTDR),  which  is  intended 
as  a  backbone  data  exchange  channel  with  reduced  cost  and  increased  capacity,  and  the 
Battlefield  Awareness  Data  Dissemination  (BADD)  System,  which  utilizes  satellite  relay 
capabilities  to  broadcast  a  broad  array  of  graphic  and  text  based  information  to  receiving 
workstations  at  all  levels.  All  of  the  communications  devices  are  interfaced  on  the  Tactical 
Internet,  a  wireless  wide  area  network.  The  ABCS  is  an  evolving  system  with  changes  in  the 
number  and  type  of  subsystems,  changes  within  subsystem  and  changes  to  network 
configurations  occurring  commonly. 

ATCCS.  The  most  familiar,  common  and  mature  of  the  ATCCS  main  subsystems  is 
MCS.  The  MCS  provides  a  facility  with  which  corps  through  battalion  commanders  and  their 
staffs  can  collect,  coordinate  and  evaluate  near  real  time  battlefield  information,  mostly  through 
graphic  representations  on  a  plan  view  display.  MCS  establishes  a  common  picture  of  enemy 
and  friendly  unit  locations,  distributes  battle  plans  and  orders,  provides  a  facility  for 
collaborative  planning  and  displays  unit  personnel  and  logistic  status  in  support  of  resource 
management  functions.  MCS  is  centered  on  a  terrain  relief  map  display  with  corps  (and  lower) 
graphic  control  measures  plus  icon  representations  of  own  force  and  opposition  unit  locations. 
Dissemination  of  plans  and  operational  orders  (OPORDs)  uses  web  site  architecture  and  a 
common  word  processing  environment  (MS-Word)  with  document  templates.  MCS  supports 
generation  of  execution  synchronization  matrices.  Collaborative  planning  is  aided  through 
interactive  graphic  input  and  a  live  audio  channel,  anticipating  geographic  dispersion  of  units 
(e.g.  a  battalion  may  be  separated  from  its  brigade  TOC  by  as  much  as  80  kilometers).  Inclusion 
of  personnel  and  logistics  status  appears  to  overlap  the  CSSCS  functions,  however,  this 
anticipates  separation  of  Administrative  and  Logistics  Operations  Center  (ALOC)  from  the 
maneuver  TOC.  MCS  will  be  located  with  corps,  division  command  staffs,  corps  signal 
battalion,  division  artillery,  mobile  and  airborne  command  vehicles,  and  at  brigade,  battalion, 
and  squadron  TOCs. 

The  AS  AS  Remote  Workstation  (RWS)  is  intended  to  display  a  combination  of  analyzed 
combat  intelligence  and  raw  combat  information.  It  provides  situational  awareness  with  regard 
to  enemy  disposition  by  integrating  multiple  information  sources  at  varying  levels  of  refinement 
including  intelligence  assessments,  processed  imagery,  raw  sensor  feeds  and  battlefield  reports. 
Processed  imagery  may  be  taken  from  national,  theater  and  tactical  sources.  Raw  sensor  sources 
may  include,  for  example,  unmanned  aerial  vehicles  (UAV)  and  the  Joint  Surveillance  Target 
Attack  Radar  System  (JSTARS).  AS  AS  provides  database  facilities  to  aid  data  collection,  data 
management  and  application  of  data  to  performance  of  Intelligence  Preparation  of  the  Battlefield 
(IPB)  functions.  In  addition,  the  AS  AS  is  the  intelligence  and  electronic  warfare  (IEW)  element 
of  the  ATCCS. 
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AFATDS  is  designed  for  the  C4I  requirements  of  fire  support  functions.  These  include 
Army  and  other  service  missile,  rocket,  cannon,  and  mortar  type  weapons  plus  close  air  support 
(CAS)  fires.  AFATDS  uses  command  guidance,  weapon  availability  and  target  type  to  match 
targets  to  optimal  weapons,  coordinates  fire  support  missions  through  distributed  messages, 
maps  and  control  graphics.  This  system  receives  message  and  sensor  information  from  Army, 
Marine  and  Air  Force  ground  based  systems.  Planning  tools  for  route  selection,  course  of  action 
analysis  and  fire  mission  planing  are  integrated. 

The  AMDW/S  generates  graphic  displays  of  weapon  and  sensor  locations,  provides  a 
means  for  altering  map  graphics,  and  facilitates  detailed  terrain  analysis,  particularly  for  line  of 
site  determination,  and  monitors  mission  progress  in  near-real  time.  AMDW/S  supports  the 
development  of  air  defense  artillery  (ADA)  missions.  For  Air  Battle  Management  (ABM),  the 
AMDW/S  provides  the  capability  to  display  air  control  orders,  current  fire  unit  status,  alert 
posture,  and  missile  expenditure.  It  is  notable  that  AMDW/S  uses  true  digitized  map  displays, 
unlike  the  other  ATCCS  main  subsystems  which  use  raster  scanned  images  of  printed  maps.  It  is 
also  notable  that  AMDW/S  would  not  normally  be  located  in  a  maneuver  battalion  TOC.  The 
major  thrust  of  AMDW/S  is  air  defense  mission  planning  and  coordination.  It  also  provides  an 
early  strike  warning  system  for  theater  ballistic  missile  attack,  including  launch  and  impact 
points.  AMDW/S  also  provides  visual  and  auditory  warning  of  approaching  enemy  aircraft  and 
automated  slew  of  designated  defensive  systems  to  counter  the  highest  threat. 

The  CSSCS  is  a  status  reporting  facility  to  aid  decision  making  and  planning  logistic 
support  of  combat  operations.  CSSCS  provides  resource  status  summary  displays,  a  course  of 
action  analysis  aid  and  correlates  supply  requirements  with  task  organization  down  to  company 
level.  CSSCS  tracks  the  Commander's  Tracked  Items  List  (CTIL),  a  subset  of  the  Baseline 
Resource  Items  List  (BRIL).  CSSCS  is  partially  integrated  with  the  more  common  Standard 
Army  Management  Information  System  (STAMIS)  but  does  not  recognize  units  below  company 
level.  The  CSSCS  is  not  normally  resident  in  a  battalion  TOC. 

There  is  one  other  main  subsystem  to  ATCCS  which  is  not  generally  presented  as  part  of 
the  suite.  This  is  the  Forward- Area  Air  Defense  Command,  Control  and  Intelligence  (FAADC2I) 
system.  Properly,  FAADC2I  includes  a  FAADC2  computer  and  software  plus  associated  sensors. 
The  FAADC2I  provides  automated  display  of  targeting  data  to  air  defense  systems  for  rapid 
threat  reaction  and  air  battle  management  in  either  engagement  operations  (EO)  or  force 
operations  (FO).  Ffence,  a  FAADC2I  terminal  may  be  referred  to  as  FAADEO.  FAADEO 
properly  refers  to  the  information  provided  to  fire  units  and  the  procedures  used  to  execute 
engagements  (i.e.  targeting).  FAADFO  properly  refers  to  air  defense  related  unit  locations  and 
battlefield  geometry  information.  The  functions  of  FAADC2I  are  similar  to  AMDW/S  with  less 
integration  of  radar  based  information. 

The  IMETS  is  a  supporting  subsystem  of  ATCCS  that  provides  current  and  predictive 
meteorological  reports  and  weather  effects  including  low  level  winds  at  specified  locations. 
IMETS  delivers  graphic  weather  overlays  to  MCS  maps  and  incorporates  the  Integrated  Weather 
Effects  Decision  Aid  (IWEDA)  for  prediction  of  weather  effects  on  weapon  systems.  The  DTSS 
is  an  automated  terrain  analysis  and  reproduction  system  for  decision  support  through  terrain 
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visualization  and  EPB.  DTSS  produces  hard  copy  color  maps  and  digitally  stored  map  images 
and  a  large  format  scanner  for  importing  nonstandard  maps.  The  CGS  is  designed  to  acquire, 
display  and  disseminate  data  from  multiple  sensors  such  as  JSTARS,  U-2,  UAVs  and  Guardrail. 

It  provides  real-time  display  of  visual  imagery  from  sensors  operating  in  a  variety  of 
electromagnetic  spectrum  bands,  as  well  as  signal  and  electronic  intelligence  acquisition 
products.  Displays  are  overlaid  on  digital  maps  and  can  provide  automated  monitoring  of 
specific  areas  of  interest.  The  BPV  system  provides  animated  course  of  action  analysis, 
wargaming,  planning,  EPB  and  mission  rehearsal  capabilities.  It  is  planned  for  integration  into 
MCS.  BPV  animates  panned  and  predicted  unit  movements  in  a  three-dimensional  terrain  model 
using  task  organization  and  friendly  unit  locations  generated  by  MCS  and  enemy  unit  locations 
generated  by  ASAS.  These  supporting  subsystems  are  not  normally  resident  at  echelons  below 
brigade. 

ATCCS  accomplishes  interoperability  through  application  of  client-server  architecture 
interfaced  by  means  of  a  web  browser  facility  (Netscape)  and  transfers  data  in  United  States 
Message  Text  Format  (USMTF)  and/or  the  File  Transfer  Protocol  (FTP).  The  client-server 
architecture  is  not  yet  folly  supported  (e.g.  AMDW/S  does  not  support  client  access).  ATCCS, 
like  ABCS,  is  an  evolving  system  with  changes  in  the  number  and  types  of  subsystems,  degree  of 
integration,  subsystem  features  network  configurations  occurring  commonly.  These  changes  are 
intended  to  increase  the  utility  of  ATCCS  within  each  of  the  five  BFAs  addressed  .  The  brief 
descriptions  presented  here  are,  to  the  best  of  the  authors'  ability,  accurate  representations  of  the 
functions  and  organization  of  the  ABCS  and  ATCCS  at  the  time  this  study  was  conducted. 

Training  Needed  to  Achieve  Information  Dominance 

Digitization  of  the  functions  and  processes  of  battle  staff  planning  and  mission  execution 
is  expected  to  provide  significant  enhancements  to  situational  awareness,  reduced  mission 
planning  time,  enhancements  to  tactical  decision  processes  and  improved  execution 
synchronization.  Increased  situational  awareness  on  the  part  of  the  commander  and  staff  is 
expected  to  improve  command  and  control  through  precision,  flexibility  and  resource 
management.  The  planning  process  is  expected  to  benefit  from  exact  duplication  of  information 
and  guidance  at  every  command  level  with  reduced  human  error  and  earlier  dissemination  of 
information  required  for  planning.  Digitally  formatted  spot  reports  and  battle  damage 
assessment  are  expected  to  be  broadcast  over  secure  networks  simultaneously  to  all  prospective 
consumers  resulting  in  greater  precision  and  timelines. 

Although  the  ATCCS  systems  have  vastly  increased  the  amount  and  currency  of  relevant 
information  that  is  available  to  a  commander  and  his  staff,  it  cannot  be  assumed  that  the 
increased  volume  of  information  will  translate  into  corresponding  improvements  in  either  the 
planning  or  the  execution  of  battlefield  operations.  Such  improvements  can  be  realized  only  if 
two  requirements  are  met.  First,  members  of  the  battle  staff  must  be  folly  knowledgeable  about 
the  capabilities  of  each  ATCCS  subsystem  and  must  be  skilled  in  using  each  system  to  plan  and 
control  battlefield  operations.  Second,  members  of  the  battle  staff  must  have  sufficient 
information  management  skills  to  avoid  becoming  overloaded  by  the  vast  variety  and  volume  of 
information  that  can  flow  from  the  ATCCS  systems. 
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Training  on  the  functions  and  operation  of  each  of  the  ATCCS  subsystems  is  necessary 
but  not  sufficient  to  ensure  information  dominance.  Battle  staff  members  also  must  receive  two 
types  of  training  on  the  collective  use  of  the  ATCCS  suite.  First,  battle  staff  members  must  be 
trained  to  use  the  suite  of  ATCCS  subsystems  as  an  integrated  system.  They  must  learn  to 
employ  the  optimal  combination  of  ATCCS  systems  to  acquire  different  information  and  to 
perform  different  tasks.  Second,  battle  staff  members  must  be  trained  to  function  as  an  efficient 
team  in  managing  information  that  flows  from  or  is  accessible  through  the  ATCCS  systems. 

Little  information  is  available  on  how  best  to  train  battle  staff  members  on  the  collective 
use  of  the  ATCCS  systems.  Although  an  ATCCS  has  been  fielded  at  the  4th  Infantry  Division 
(4ID),  Fort  Hood,  Texas,  there  has  been  little  time  to  develop  and  assess  a  method  for  training 
battle  staff  members  on  the  collective  use  of  the  ATCCS.  The  research  literature  is  replete  with 
reports  on  team  training,  for  example  see  the  bibliography  by  Salas  (Salas,  1997),  but  only  one 
document  has  been  located  that  describes  research  aimed  specifically  at  information  management 
by  a  team  in  a  command  and  control  setting  (Freeman,  Cohen,  Serfaty,  Thompson,  &  Bresnick, 
1997).  This. document  describes  an  excellent  exploratory  research  project  that  was  designed  to 
identify  (a)  fundamental  information  management  principles  and  procedures  and  (b)  methods  for 
training  battle  staff  members  to  apply  the  information  management  principles  and  procedures. 
Freeman  and  his  associates  also  discuss  methodological  problems  that  complicate  the 
investigation  of  information  management,  including  problems  in  assessing  the  effectiveness  with 
which  information  is  managed.  Although  the  report  by  Freeman  and  his  associates  (Freeman  et 
al.,  1997)  contains  a  great  deal  of  useful  information,  the  authors  acknowledge  that  their  work 
represents  only  a  first  step  in  a  program  of  research  that  is  needed  to  develop  and  validate 
methods  to  train  battle  staff  members  to  manage  information  effectively. 

Anticipating  the  delivery  of  ATCCS  to  Army  aviation  battalions,  there  is  a  pressing  need 
to  develop  effective  methods  for  training  the  battle  staffs  of  Army  aviation  battalions  on  the 
collective  use  of  the  ATCCS  systems,  battalion  battle  staff  members  must  be  trained  on  the 
optimal  use  of  ATCCS  systems  for  both  planning  and  executing  missions.  Training  on 
information  management  principles  and  methods  must  be  considered  an  integral  part  of  training 
on  the  collective  use  of  the  ATCCS  systems. 

A  program  of  research  is  needed  to  develop  and  validate  methods  for  training  battle  staff 
members  on  the  collective  use  of  ATCCS  systems.  An  essential  first  step  in  this  program  of 
research  is  to  develop  methodology  for  research  to  (a)  identify  potential  training  methods  and  (b) 
assess  the  relative  training  and  cost  effectiveness  of  alternative  training  methods.  It  is  essential 
that  methodology  be  developed  to  assess  the  information  management  performance  of  battle  staff 
members,  individually  and  collectively. 


Rationale 

This  section  describes  the  rationale  underlying  the  research  approach  and  methods.  This 
rationale  is  heavily  influenced  by  the  results  of  prior  research  on  team  performance,  especially 
research  on  the  performance  of  military  command  and  control  teams  (Brannick,  Salas,  and 
Prince,  1997;  Guzzo  and  Salas,  1995).  The  rationale  is  also  influenced  by  research  on 
naturalistic  decision  making,  which  has  been  conducted  to  gain  a  better  understanding  of  how 
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decisions  are  made  in  a  complex,  dynamic  environment  such  as  a  command  and  control  center 
(Zsambok  and  Klein,  1997). 

This  study  was  not  intended  to  be  a  classical  experiment  in  which  independent  variables 
are  varied  systematically.  A  classical  experiment  was  judged  unsuitable  for  two  reasons.  The 
most  practical  reason  is  that  the  time  and  resources  available  were  insufficient  to  conduct  a 
productive  experiment.  More  importantly,  an  experiment  is  not  the  most  efficient  approach  for 
accomplishing  the  objectives  of. this  study.  So  little  is  known  about  how  best  to  use  digital 
equipment  in  an  Army  aviation  C4I  facility  that  it  was  not  possible  to  identify,  a  priori,  a  set  of 
independent  variables  that  would  be  expected  to  determine  observable  behaviors  in  a  controlled 
experiment. 

Focus  on  Teamwork 

A  fundamental  assumption  underlying  the  research  plan  is  that  the  observations  should 
emphasize  team  behaviors  (i.e.,  teamwork)  over  individual  behaviors  (i.e.,  taskwork).  A 
considerable  amount  is  known  about  how  to  train  battle  staff  members  in  their  functional-areas 
and  about  how  to  train  digital  equipment  operators  to  perform  their  individual  tasks.  However, 
little  is  known  about  how  best  to  train  battle  staff  members  and  digital  equipment  operators  to 
function  effectively  as  an  integrated  team  in  a  digital  C4I  facility.  Acquiring  knowledge  about 
the  behaviors  that  lead  to  effective  teamwork  is  an  essential  first  step  in  developing  effective 
TTPs  and  effective  team  training  methods. 

The  focus  on  teamwork  should  not  imply  that  the  individual  training  of  either  the  battle 
staff  members  or  the  equipment  operators  is  adequate.  To  the  contrary,  survey  data  compiled 
recently  (Thompson,  Thompson,  Pleban,  and  Valentine,  1991)  show  that  most  junior  Cavalry 
and  Infantry  officers  have  had  little  or  no  functional-area  training  at  the  time  they  are  assigned  to 
a  battalion  battle  staff  position.  Similarly,  there  is  some  anecdotal  evidence  that  many  of  the 
individuals  who  are  expected  to  operate  the  digital  equipment  in  a  digitized  TOC  are  not 
adequately  trained.  Individual  skills  are  important  to  the  extent  that  deficiencies  make  it  difficult 
or  impossible  to  draw  valid  inferences  about  the  behaviors  that  lead  to  effective  team 
performance. 

Types  of  Measures  Required 

Two  types  of  performance  measures  were  needed  to  accomplish  the  objectives  of  this 
study:  outcome  measures  and  process  measures.  As  the  name  implies,  outcome  measures  reflect 
the  quality  or  desirability  of  the  intermediate  or  final  result  of  the  mission  exercises.  Process 
measures  reflect  the  type  and  quality  of  the  activities,  strategies,  responses,  and  behaviors  that 
are  employed  by  the  battle  staff  (Cannon-Bowers  and  Salas,  1997;  Coovert,  Cannon-Bowers,  and 
Salas,  1990). 

Outcome  measures  are  important  but  are  not  fully  adequate  for  present  purposes.  One 
reason  is  that  outcome  measures  are  not  diagnostic;  they  provide  no  insight  about  why  the 
outcome  was  favorable  or  not  favorable.  Another  reason  is  that  desirable  outcomes  may  be  the 
result  of  chance  or  simply  good  luck,  rather  than  effective  performance  by  the  battle  staff.  In 
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such  instances,  drawing  inferences  only  from  the  outcome  measures  might  lead  to  the 
reinforcement  of  ineffective  behavior. 

Process  measures  are  needed  to  understand  the  causal  relationships  between  various 
performance  elements  (i.e.,  activities,  strategies,  responses,  and  behaviors)  and  outcomes.  The 
processes  of  interest  for  this  study  are  the  tactics,  techniques,  and  procedures  (TTPs)  employed 
by  the  battle  staff.  Only  when  key  processes  are  well  understood  is  it  possible  to  specify  the 
behaviors  that  lead  to  desired  outcomes  and  the  behaviors  that  do  not.  The  processes  that  are 
considered  important  for  this  study  are  discussed  below. 

Key  Processes 

Although  there  are  some  processes  that  appear  to  be  common  to  most  types  of  teams,  it  is 
generally  agreed  that  the  processes  underlying  effective  team  performance  differ  to  from  one 
type  of  team  to  another.  In  identifying  processes  relevant  to  this  study,  we  defined  the  following 
as  the  distinguishing  characteristics  of  a  battle  staff  viewed  as  a  team: 

'  •  the  team  has  a  hierarchical  structure; 

•  the  individual  responsibilities  and  duties  of  team  members  are  reasonably  well 
defined  and  stable, 

•  team  members’  tasks  tend  to  be  interdependent, 

•  there  is  a  relatively  small  amount  of  overlap  in  the  responsibilities  and  duties  of 
team  members, 

•  the  large  majority  of  both  individual  and  team  tasks  involve  the  acquisition, 
processing,  or  distribution  of  information, 

•  the  overriding  team  goal  is  to  provide  the  information  that  the  commander  needs  to 
assess  the  situation  continuously  and  accurately,  and 

•  the  situation  assessment  must  enable  the  commander  to  make  rapid  and  tactically 
sound  decisions  about  the  need  for  intervention  in  the  planned  mission  and  the  type 
of  intervention  that  is  required. 

The  processes  described  below  are  named  and  defined  in  a  manner  that  is  considered 
most  meaningful  for  the  study  of  teamwork  among  members  of  a  battle  staff.  Virtually  all  of 
these  processes  have  been  identified  and  described  by  team  performance  researchers,  albeit  with 
varied  terminology. 

The  processes  that  are  causally  related  to  the  performance  of  a  command  and  control 
team  can  be  defined  at  different  levels  of  generality.  At  the  most  general  level,  we  have  defined 
three  key  processes:  situation  assessment,  decision  making,  and  workload  management,  these 
processes  can  be  thought  of  as  “first  tier”  processes  because  they  have  a  direct  and  immediate 
impact  on  mission  outcome.  At  a  lower  level  of  specificity  are  the  teamwork  processes  that  are 
causally  related  to  the  first  tier  processes.  The  teamwork  processes  can  be  thought  of  as  second 
tier  processes  because  they  influence  mission  outcome  through  one  or  more  of  the  first-tier 
processes.  The  first-tier  and  second  tier  processes  are  discussed  in  the  following  paragraphs. 
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Establishment  and  sustainment  of  situation  awareness.  During  the  past  10  years,  a 
state  of  cognitive  awareness,  referred  to  as  situation  awareness,  has  been  found  to  have  profound 
influence  on  the  performance  of  a  variety  of  complex  tasks,  including  decision  making  in  a 
command  and  control  facility  (Endsley,  1997).  A  recurrent  problem  in  situation  awareness 
research  has  been  definition  of  the  concept;  several  have  been  used.  The  following  definition, 
though  academic,  provides  broad  applicability  and  pathways  to  observable  behavior. 

“Situation  awareness  is  the  perception  of  the  elements  in 
the  environment  within  a  volume  of  time  and  space,  the 
comprehension  of  their  meaning  and  the  projection  of  their 
status  in  the  near  future  (Endsley,  1988). 

This  definition  implies  three  levels  of  situation  awareness.  Level  1  is  perception  of 
critical  elements  in  the  environment;  Level  2  is  understanding  the  factors’  meaning  when 
interpreted  in  light  of  the  goal;  Level  3  is  understanding  what  will  happen  in  the  near-term 
future.  We  believe  that  Endsley’s  definition  is  suitable  for  the  present  purposes. 

When  monitoring  the  progress  of  a  tactical  mission,  the  situation  can  change  at  any 
moment.  Hence,  the  process  of  establishing  and  maintaining  situation  awareness — situation 
assessment — is  a  continuous  process  for  which  intensity  (and  by  inference,  resource 
requirements)  may  vary  considerably  from  one  mission  segment  to  another.  Although  it  is  true 
that  the  commander’s  situation  awareness  has  the  most  direct  influence  on  decisions  to  intervene, 
the  process  of  situation  assessment  is  a  team  effort.  That  is,  every  member  of  the  command  and 
control  team  is  responsible  for  seeking  and  transmitting  information  that  contributes  to  the 
commander’s  situation  awareness. 

Tactical  decision  making.  Tactical  decision  making  has  long  been  recognized  as  the 
process  that  has  the  most  direct  and  immediate  impact  on  command  and  control  effectiveness. 
However,  it  is  only  in  the  last  decade  that  meaningful  research  has  been  conducted  to  better 
understand  how  decisions  are  made  in  a  dynamic,  uncertain,  and  often  fast  paced  environment 
such  as  a  command  and  control  facility.  The  research  has  shown  that  decisions  in  such  situations 
are  not  made  in  accordance  with  the  classical  decision-making  paradigm  in  which  multiple 
options  are  identified  and  compared  with  one  another.  Rather,  the  research  findings  led  to  the 
identification  of  a  Naturalistic  Decision  Making  (NDM)  process  (Zsambok  and  Klein,  1997).  A 
highly  simplified  concept  of  the  NDM  process  is  that  the  decision  maker  identifies  the  situation 
as  familiar  and  recalls  a  course  of  action  from  memory)  that  is  appropriate  for  such  situations. 
The  NDM  process  is  aimed  at  identifying  the  first  course  of  action  that  will  work  rather  than 
identifying  the  optimal  course  of  action  among  many  options. 

When  evaluating  the  effectiveness  of  the  battle  staff,  it  certainly  is  important  to  assess  the 
timeliness  and  tactical  suitability  of  the  decisions  that  are  made.  However,  it  is  equally 
important  to  gain  a  better  understanding  of  the  cognitive  processes  through  which  decisions  are 
made. 


Workload  management.  Perhaps  the  greatest  concern  about  the  development  and  use  of 
digitized  C4I  systems  is  that  the  workload  associated  with  processing  the  information  will,  at. 
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times,  exceed  the  capacity  of  one  or  more  members  of  the  command  and  control  team.  This  is  a 
valid  concern.  The  benefits  of  the  enormous  increase  in  information  availability  could  be  offset 
by  the  errors  and  inefficiencies  that  are  the  typical  result  of  prolonged  exposure  to  excessive 
workload.  It  is  for  this  reason  that  workload  management  is  expected  to  be  influential  with 
respect  to  effectiveness  of  battle  staff  performance.  As  is  discussed  below,  reallocating  functions 
among  team  members  is  one  way  to  manage  workload.  Examples  of  other  coping  strategies 
include  dropping  a  task  altogether,  attending  to  tasks  in  the  order  of  their  priority,  accepting  a 
delay  in  responding  to  task  demands,  and  exerting  more  effort. 

Teamwork  Processes 


The  processes  described  below  are  named  and  defined  in  a  manner  that  is  considered 
most  meaningful  for  the  study  of  teamwork  among  members  of  a  battle  staff.  Virtually  all  of 
these  processes  have  been  identified  and  described  by  team  performance  researchers,  albeit  with 
somewhat  different  names  and  definitions,  and  all  have  been  found  to  have  a  major  influence  on 
the  effectiveness  of  team  performance  under  some  circumstances. 

Establish/maintain  shared  mission  concept.  Problems  encountered  during  mission 
execution  are  sometimes  explained  by  stating  that  “not  all  participants  were  playing  from  the 
same  sheet  of  music.”  Researchers  in  team  performance  express  the  same  idea  by  stating 
“participants’  mental  models  of  the  mission  differed  in  important  respects.”  Both  phrases 
accurately  convey  the  notion  that  the  participants’  understanding  of  one  or  more  characteristics 
of  the  mission  differed  enough  to  create  problems.  The  term  “shared  mission  concept”  is  used 
here  to  describe  this  important  requirement  for  effective  teamwork.  More  needs  to  be  learned 
about  (a)  the  types  of  problems  that  are  caused  by  the  lack  of  a  shared  mission  concept  and  (b) 
the  methods  whereby  a  common  mission  concept  can  be  established  and  maintained  for  all 
members  of  the  battle  staff. 

Establish/maintain  shared  concept  of  team  organization.  Team  performance  can  be 
seriously  degraded  by  misunderstandings  about  the  organization  of  a  team.  It  is  particularly 
important  for  team  members  to  have  a  common  understanding  of  (a)  proper  communication 
channels  and  (b)  the  duties  and  responsibilities  of  every  team  member.  Establishing  and 
maintaining  a  shared  concept  of  team  organization  would  be  a  fairly  simple  matter  if  the  team 
organization  remained  constant  over  time.  Battle  staff  organization  may  be  expected  to  vary  as  a 
function  of  the  commander’s  style  of  operation.  However,  it  is  likely  that  the  team  organization 
changes  over  time  as  a  result  of  situational  factors  and  team  members’  competencies. 

Information  is  needed  about  (a)  the  extent  to  which  team  organization  changes  over  time,  (b)  the 
reasons  for  the  changes,  (c)  the  team  members’  perception  of  the  changes,  and  (d)  the  best  ways 
to  establish  and  maintain  a  common  concept  of  team  organization. 

Information  screening.  One  of  the  concerns  about  digital  C4I  systems  is  the  mass  of 
information  that  is  available  to  every  equipment  operator  and  battle  staff  member.  A  heavy  flow 
of  information  imposes  a  requirement  to  filter  the  flow  such  that  the  information  conveyed  from 
one  team  member  to  another  is  relevant  for  the  mission  and  relevant  for  the  recipient  of  the 
information.  There  is  no  doubt  that  effective  information  screening  is  necessary  to  avoid  (a) 
overloading  the  commander  or  some  other  team  member  with  an  excessive  amount  of 
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nonessential  information  or  (b)  denying  the  commander  critically  important  information. 
However,  we  have  found  no  useful  guidance  about  who  should  be  responsible  for  screening 
information  (equipment  operators,  battle  staff  members,  or  both)  or  how  to  train  personnel  to 
screen  information  effectively. 

Information  pushing.  An  important  characteristic  of  an  effective  team  is  that  team 
members  have  a  clear  understanding  of  the  information  that  is  important  to  another  team  member 
and  are  sufficiently  assertive  to  communicate  the  information  in  a  timely  and  forceful  manner. 
This  may  mean  interrupting  a  superior  in  order  to  “push”  the  information  to  him  or  her.  Many 
catastrophic  events  have  occurred  because  a  team  member  lacked  the  knowledge  or  assertiveness 
to  “push”  important  information  to  another  team  member,  so  it  is  reasonable  to  assume  that  such 
problems  occur  in  military  command  and  control  facilities  such  as  a  digitized  TOC. 

Information  seeking.  It  is  seldom,  perhaps  never,  possible  to  access  all  the  information 
that  is  needed  to  continuously  assess  the  situation  through  passive  monitoring  alone.  The  digital 
information,systems  are  designed  such  that  important  information  is  presented  on  different 
“pages”  or  different  display  modes.  Hence,  it  is  necessary  for  an  equipment  operator  or  a 
member  of  the  battle  staff  to  recognize  information  deficiencies  and  to  acquired  the  information 
through  database  searches,  page  changes,  display  mode/format  changes,  or  directing  another 
person  to  acquire  the  needed  information.  The  need  to  actively  seek  information  may  be 
stimulated  by  a  recognition  that  (a)  important  information  has  aged  and  needs  to  be  updated  or 
(b)  events  have  occurred  that  cannot  be  evaluated  fully  with  the  information  available.  In 
principle,  the  search  for  additional  information  can  be  initiated  by  an  equipment  operator  or  by 
any  member  of  the  battle  staff.  In  practice,  however,  little  is  known  about  the  cues  that  signal 
the  need  to  seek  additional  information  or  who  can  and  should  be  responsible  for  recognizing 
such  cues. 

Mutual  monitoring.  A  team  characteristic  that  appears  to  be  important  for  every  type  of 
team  is  the  capability  and  inclination  of  team  members  to  monitor  the  performance  of  other  team 
members  and  to  alert  other  team  members  of  errors  of  omission  and  commission.  Mutual 
monitoring  is  particularly  important  for  teams  that  are  required  to  monitor  a  continuous  stream  of 
information.  The  equipment  and  personnel  in  the  digitized  TOC  provide  the  capability  for  some 
amount  of  mutual  monitoring.  For  example,  members  of  the  battle  staff  can  view  the  monitors 
used  by  the  equipment  operators  and  the  displays  on  the  “video  wall.”  More  needs  to  be  learned 
about  the  amount  and  type  of  mutual  monitoring  that  is  feasible  and  beneficial. 

Coordination.  As  mentioned  earlier,  a  distinguishing  characteristic  of  battle  staff  is  the 
task  interdependency  among  the  team  members.  That  is,  there  are  tasks  that  one  team  member 
cannot  perform  without  information  that  must  be  provided  by  another  team  member.  In  such 
cases,  job  performance  by  a  team  member  can  be  seriously  degraded  if  information  needed  from 
another  team  member  is  delayed  or  not  provided  at  all.  To  the  extent  that  team  members’  tasks 
are  interdependent,  team  performance  will  be  heavily  dependent  on  the  degree  of  coordination  in 
the  exchange  of  information. 

Backup  behavior.  For  many  types  of  teams,  the  workload  of  team  members  is  highly 
variable  and  event  dependent.  The  performance  of  the  entire  team  can  suffer  when  a  team  . 
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member’s  workload  becomes  excessive.  A  characteristic  of  an  effective  team  is  the  ability  and 
inclination  of  one  team  member  to  help  another  team  member  whose  workload  has  become 
excessive.  Coming  to  the  aid  of  a  team  member  is  referred  to  here  and  elsewhere  as  “backup 
behavior.”  Because  backup  behavior  is  recognized  as  an  important  contributor  to  team 
performance,  there  is  a  need  to  learn  more  about  the  potential  for  promoting  more  backup 
behavior  in  the  digitized  TOC. 

Dynamic  reallocation  of  functions.  Backup  behavior,  discussed  above,  is  one  way  to 
cope  with  excessive  workload  of  a  team  member.  Another  way  is  for  the  team  leader  to  direct  a 
change  in  the  allocation  of  functions  (viz.,  responsibilities  and  duties)  among  two  or  more  team 
members.  Team  performance  under  heavy  workload  conditions  is  closely  related  to  the  team 
members’  capability  to  adapt  quickly  and  effectively  to  a  reallocation  of  functions.  A  battle  staff 
leader’s  (commander  or  another  member  of  the  battle  staff)  options  for  dynamic  reallocation  of 
functions  are  largely  determined  by  the  degree  of  cross  training  by  members  of  the  battle  staff. 

Objectives 

The  broad  objective  of  this  study  was  to  compile  the  data  needed  to  address  the 
requirements  listed  at  the  beginning  of  this  introductory  section.  However,  the  specific 
objectives  of  this  study  were  influenced  by  three  other  factors.  First,  the  specific  objectives  were 
influenced  by  constraints  imposed  by  (a)  limited  time,  equipment,  and  personnel  and  (b)  the 
requirement  to  use  data  collection  methods  that  did  not  interfere  with  the  training  of  the  Aviation 
Officers  Advanced  Course  (AVOAC)  students.  The  constraints  prevented  the  design  and 
conduct  of  a  classical  experiment  in  which  performance  is  measured  as  conditions  (variables)  are 
varied  systematically.  As  a  consequence,  it  was  necessary  to  conduct  an  “observational  study”  in 
which  behaviors  and  outcomes  are  observed  and  inferences  are  drawn  about  the  behaviors 
(processes)  that  lead  to  desirable  outcomes  and  those  that  lead  to  undesirable  outcomes.  It  was 
also  necessary  to  employ  survey  instruments  and  interviews  to  obtain  information  from  the 
individuals  who  served  as  battle  staff  members  during  the  study. 

Second,  the.  specific  objectives  were  influenced  by  the  lack  of  a  proven  methodology  for 
quantifying  and  assessing  information  management  methods  and  procedures.  The  observation 
methods  that  were  employed  were  based  on  the  assumption  that  an  assessment  of  information 
management  must  focus  on  the  type  and  volume  of  information  that  flows  to  and  from  each 
member  of  the  battle  staff,  particularly  the  commander. 

Third,  the  specific  objectives  were  influenced  by  the  need  to  identify  exercise-design 
problems  that  must  be  taken  into  account  when  interpreting  the  findings.  Unrealistic  mission 
scenarios,  inadequate  mission  planning  resources  (e.g.,  inadequate  time,  information,  personnel), 
requirement  to  use  nondoctrinal  tactics  are  examples  of  potential  exercise-design  problems  that 
were  considered  when  designing  this  study. 

Described  below  are  the  specific  objectives  established  for  the  compilation  of  data  about 
the  events  that  occurred  and  the  problems  that  were  encountered  during  mission  planning  and 
during  mission  execution. 
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Objectives  of  Mission  Planning  Observations/Surveys 

The  objectives  listed  below  influenced  the  compilation  of  information  about  mission 
planning  activities  and  problems.  To  the  extent  possible,  the  study  was  designed  to  provide 
detailed  information  about  the  performance  of  each  MDMP  step.  So,  many  of  the  objectives 
listed  below  apply  to  MDMP  steps  considered  individually  and  collectively. 

•  Determine  the  manner  in  which  the  ATCCS  systems  are  used  during  mission 
planning. 

•  Identify  the  problems  that  are  encountered  in  using  the  ATCCS  systems  during 
mission  planning. 

•  Determine  the  battle  staff  members  who  participate  in  each  MDMP  step/substep  and 
assess  the  relative  workload  of  the  battle  staff  members. 

•  Determine  the  time  spent  on  each  MDMP  step. 

•  Determine  the  adequacy  of  the  time  available  for  mission  planning. 

•  Determine  the  adequacy  of  the  information  available  for  mission  planning  (type  and 
accessibility  of  information). 

•  Determine  the  difficulty  and  realism  of  the  mission  being  planned. 

•  Identify  the  need  to  establish/modify  TTPs  that  govern  mission  planning  activities. 

•  Identify  the  need  for  and  function  of  an  information  manager  (to  manage  information 
during  mission  planning). 

•  Determine  the  need  for  increased  automation  and  identify  the  planning 
tasks/functions  for  which  increased  automation  is  needed. 

•  Identify  knowledge  and  skill  deficiencies  that  adversely  influence  mission  planning 
quality  and/or  efficiency. 

•  Assess  the  observation  methodology  and  the  survey  instruments. 

Objectives  of  Mission  Execution  Observations/Surveys 

The  objectives  listed  below  influenced  the  compilation  of  information  about  mission 
execution  activities  and  problems.  Many  of  the  objectives  are  related  to  the  measurement  and 
assessment  of  information  flow  to  and  from  the  members  of  the  battle  staff  and  other  battlefield 
elements. 

•  Determine  the  manner  in  which  the  ATCCS  systems  are  used  during  mission 
execution  to  acquire  and  disseminate  information. 

•  Identify  the  problems  that  are  encountered  in  using  the  ATCCS  systems  during 
mission  execution. 

•  Quantify  the  type  and  volume  of  information  that  flows  to  and  from  members  of  the 
battle  staff,  with  special  emphasis  on  the  information  flow  to  and  from  the 
commander. 
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Determine  the  frequency  with  which  the  commander  acquires  information  through  the 
supply-push  method  or  the  demand-pull  method.1 

Identify  the  incidence,  consequences,  and  causes  of  excessive  information  processing 
workload. 

Assess  the  utility  of  a  wall-mounted  video  display2  for  disseminating  information  to 
the  commander  or  to  other  members  of  the  battle  staff. 

Identify  the  need  to  establish/modify  TTPs  that  govern  (a)  monitoring  the  execution 
of  the  decision  support  template,  (b)  the  management  of  information,  and  (c)  other 
mission  execution  activities. 

Identify  the  need  for  and  functions  of  an  information  manager  (to  manage  information 
during  mission  execution). 

Identify  knowledge  and  skill  deficiencies  that  adversely  influence  the  quality  of  the 
mission  outcome  or  the  efficiency  of  the  mission  execution  activities. 

Assess  the  observational  methodology  and  the  survey  instruments. 

Method 

Facilities  and  Equipment 

This  study  was  conducted  in  two  major  facilities:  the  Army  Aviation  Test  Bed3  (AVTB) 
and  the  Aviation  Warfighting  Simulation  Center  (AWSC).  The  AVTB  and  AWSC  are 
interconnected  with  a  fiber  optic  cable  network;  both  facilities  are  located  at  the  USAAVNC, 

Fort  Rucker,  Alabama.  The  AVTB  and  the  AWSC  are  simulation  facilities  that  can  be 
reconfigured  to  support  a  wide  range  of  Army  aviation  research.  The  configurations  employed 
for  this  study  are  described  below. 

Brigade  and  Battalion  TOCs 

Both  the  brigade  TOC  (located  in  the  AVTB)  and  battalion  TOC  (located  in  the  AWSC) 
were  configured  as  simulated  digitized  TOCs  using  a  partial  installation  of  the  ATCCS  suite. 

The  brigade  TOC  served  as  a  control  center  during  this  study.  All  the  observational  data  that 
were  compiled  during  this  study  result  from  the  observations  of  events  and  activities  that 
occurred  in  the  battalion  TOC.  The  brigade  and  battalion  TOCs  each  had  three  cells:  a  current 
operations  cell,  a  plans  cell,  and  an  ALOC.  Each  of  the  three  cells  is  described  below. 

Current  operations  cells.  The  current  operations  cells  were  designed  to  accommodate 
the  battle  staff  members  who  participated  in  the  execution  of  missions  and  support  their 
functions.  During  the  period  between  missions,  mission  planning  discussions  sometimes  took 
place  in  the  current  operations  cells,  but  virtually  all  mission  planning  tasks  were  performed  in 


1  The  supply-push  and  the  demand-pull  methods  of  acquiring  infonnation  are  defined  and  discussed  in  the  Findings 
section  of  the  report  under  the  heading  “Information  Flow/Management.”  For  present  purposes,  demand-pull 
means  that  the  information  was  conveyed  as  a  result  of  an  overt  request  for  the  information.  Supply-push  means 
that  infonnation  was  conveyed  to  an  individual  without  it  being  overtly  requested  by  that  individual. 

2  As  discussed  in  a  later  section  of  tire  report,  the  wall-mounted  video  display  is  capable  of  portraying  information 
from  any  of  the  ATCCS  systems. 

3  The  AVTB  was  formerly  known  as  AIRNET. 
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the  plans  cells.  Both  current  operations  cells  were  equipped  with  the  digital  systems  that  are 
listed  below.  Note  that  the  last  two  are  not  workstations  but  passive  information  displays. 

•  ASAS 

•  MCS 

•  AFATDS 

•  AMDW/S 

•  FAADEO 

•  JSTARS  -  MTI.  JSTARS  is  a  wide  area  airborne  surveillance  system  that  provides 
battle  management  and  targeting  information  to  the  commander  and  staff  It  detects 
moving  and  fixed  targets  with  accuracy  that  afford  precision  strike  engagements  at 
extended  ranges.  The  JSTARS  Moving  Target  Indicator  (MTI),  which  presents  a 
symbolic  depiction  of  the  moving  objects  present  in  the  ground  area  covered  by  the 
display. 

•  Tactical  Unmanned  Aerial  Vehicle  (TUAV)  Display — The  TUAV  display  presents 
raw  video  imagery  of  terrain  and  ground-based  targets  as  seen  from  an  airborne 
TUAV  as  modeled  through  a  simulation-based  interactive  eyepoint. 

In  addition  to  the  suite  of  digital  systems  described  above,  the  operations  cells  were 
equipped  with  a  radio  system  emulator  and  a  ’’video  wall.”  The  16-channel  radio  system 
emulator,  produced  by  Advanced  Simulation  Technology,  Inc.  (ASTi),  was  capable  of 
simulating  the  line-of-sight  and  range  limitations  of  operational  FM  radios.  Because  headphones 
were  not  available,  voice  messages  were  heard  on  small  speakers  located  at  most  but  not  all 
workstations  in  the  current  operations  cells. 

The  video  wall  contained  a  large  screen  display  (35  inch  diagonal)  placed  44  inches 
above  the  floor.  This  was  surrounded  on  three  sides  (top,  left,  and  right)  by  eight  small  screen 
displays  (17  inch  diagonal).  The  video  wall  displays  were  connected  to  a  switching  system  that 
enabled  operators  to  display  the  AFATDS  output,  the  JSTARS  output,  the  TUAV  video,  the 
ASAS  output,  or  the  AMDW/S  output  on  any  one  of  six  small  screen  displays4  or  on  the  large 
screen  display. 

Plans  cells.  The  plans  cells  in  the  brigade  battalion  TOCs  were  designed  to  serve  as 
future  operations  planning  areas  for  use  by  the  members  of  the  battle  staff  who  participated  in 
mission  planning.  In  addition  to  work  tables,  map  boards,  and  other  mission  planning  materials, 
the  plans  cells  were  equipped  with  an  ASAS  Remote  Work  Station  (RWS)  and  an  MCS  station. 
Both  the  ASAS-RWS  and  the  MCS  station  were  capable  of  transmitting  data  to  and  receiving 
data  from  the  current  operations  cells  via  tactical  Internet. 

ALOC  cells.  The  ALOC  cells  were  designed  to  accommodate  the  personnel  responsible 
for  the  planning  and  coordination  of  combat  service  support  (CSS)  and  facilitate  their  functions. 
Both  ALOC  cells  were  equipped  with  an  MCS  and  CSSCS  workstation.  Voice  communication 
between  the  ALOC  cells  and  the  current  operations  cells  was  accomplished  with  the  radio  system 
emulator. 

4  Two  of  the  eight  small  screen  displays  were  spares. 
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Aircraft  Simulators 

This  study  employed  six  Simulation  Network  (SIMNET)  Fully  Reconfigurable 
Experimental  Devices  (FREDs)5  located  in  the  AVTB.  Three  FREDs  were  configured  as  AH- 
64A  aircraft,  two  were  configured  as  OH-58D  aircraft,  and  one  was  configured  as  a  UH-60 
aircraft.  The  six  FREDs  were  networked  together  and  networked  with  the  simulation 
components  described  in  the  next  subsection.  All  FREDs  had  virtual  (out-the-window)  displays 
that  were  generated  from  the  same  digital  topography  (Grafenfels  database).  Each  FRED  was 
occupied  by  a  company  commander  and  a  pilot  or  by  two  pilots.  Each  was  connected  to  the 
radio  emulator  network. 

Company  TOC 

An  area  in  the  AVTB  was  designated  as  a  company-level  TOC.  The  company  TOC  was 
used  by  the  company  commanders  and  pilots  to  plan  their  missions.  The  company  TOC  was  not 
occupied  during  the  time  that  missions  were  being  conducted.  The  company  TOC  was  equipped 
with  maps  and  other  planning  materials,  but  was  not  equipped  with  any  digital  system. 

Simulation/Interface  Components  and  Systems  Modeled 

The  ATCCS  digital  subsystems  and  displays  and  the  FREDs  received  tactical  message 
inputs  from  constructive  and  virtual  interfaces  through  a  distributed  interactive  simulation  (DIS) 
local  area  network  (LAN)  and  wide  area  network  (WAN).  The  tactical  messages,  in  turn,  were 
determined  by  a  synthetic  battlespace  environment  that  was  generated  by  the  simulation 
components  and  interface  components  listed  below. 

•  Computer  Generated  Forces:  Modular  Semi- Automated  Forces  (CGF  ModSAF) 

•  Tactical  Simulation  Interface  Unit  (TSIU) 

•  Extended  Air  Defense  Simulation  (EADSIM) 

•  TUAV  Simulation  (MetaVR) 

•  Protocol  Interface  Unit  (PIU) 

•  Target  Acquisition  Fire  Support  Model  (TAFSM) 

•  Sensor/C4I  Simulation 

The  simulation  and  interface  components  used  for  this  study  had  the  capability  to 
simulate  (model)  both  friendly  (blue)  and  enemy  (red)  maneuver  forces  and  aircraft  (fixed-wing 
and  rotary-wing).  In  addition,  the  simulation  components  listed  above  had  the  capability  to 
generate  an  airborne  visual  sensor  and  the  following  weapons  systems: 

•  Army  Tactical  Missile  System/Multiple  Launched  Rocket  System 
(ATACMS/MLRS), 

•  Phased  Array  Tracking  Intercept  of  the  Target  (PATRIOT)  Missiles, 

•  Cruise  Missiles,  and 

•  Threat  Transporter-Erector-Launcher  (TEL)  Missiles. 


5  A  FRED  is  a  fixed-base  aircraft  simulator  that  can  be  reconfigured  to  simulate  different  types  of  rotary-wing 
aircraft  or  semi-permanently  to  represent  a  fixed-wing  aircraft. 
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After  Action  Review  (AAR1  Area 

A  briefing  room,  located  in  the  AWSC,  was  used  to  conduct  AARs.  This  room  had  a 
large  enough  seating  area  to  accommodate  all  the  personnel  who  were  involved  in  the  study.  It 
was  equipped  with  three  front  projection  screens  and  a  suite  of  projectors  for  displaying  slides, 
transparencies,  and  video  imagery.  The  video  projector  was  used  to  project  imagery  from  a 
video  recorder  and  imagery  from  a  computer  (viz.,  Powerpoint  slides).  In  addition,  the  display 
facilities  of  the  AAR  room  could  be  fed  imagery  from  various  sources  within  the  battalion  TOC 
ATCCS  suite. 

Personnel 

Three  categories  of  personnel  provided  support  for  this  study.  One  category  was  the 
company  commanders  and  pilots  who  performed  the  company-level  mission  planning  and  who 
occupied  the  FREDs  during  the  mission  executions.  The  second  category  included  the  personnel 
who  served  .as  members  of  the  battalion  battle  staff.  The  third  group  included  the  personnel  who 
served  as  members  of  the  research  support  staff.  Each  of  the  three  groups  of  personnel  is 
described  below. 

Company  Commanders  and  Pilots 

Three  individuals  served  as  commanders  of  attack  helicopter  companies  during  this 
study.  A  company  commander  occupied  each  of  the  three  FREDs  that  were  configured  as  an 
AH-64A  aircraft.  The  three  individuals  who  served  as  company  commanders,  all  qualified  Army 
aviators,  were  in  the  process  of  completing  AVOAC,  class  98-1,  at  the  time  the  study  was 
conducted.  Nine  individuals  who  served  as  FRED  pilots  (3  AH-64As,  4  OH-58Ds,  and  2  UH- 
60s)  were  casual  officers  assigned  to  the  First  Battalion  of  the  145th  Aviation  Regiment  (1/145), 
First  Aviation  Brigade;  all  were  qualified  Army  aviators. 

Battalion  Battle  Staff 

A  group  of  12  Army  aviators  served  as  battalion  battle  staff  members  during  all  or  a  part 
of  this  study.  Eleven  of  the  aviators  were  in  the  process  of  completing  the  AVOAC,  class  98-1. 
One  aviator  was  a  company  commander  in  the  1/145  at  the  time  they  participated  in  this  study. 

At  the  outset  of  the  study,  an  individual  was  assigned  to  each  of  the  following  battle  staff 
positions: 


•  Commanding  Officer(CO), 

•  Executive  Officer  (XO), 

•  Battle  Captain  (BC), 

•  Assistant  Battle  Captain  (ABC), 

•  Personnel  Officer  (SI), 

•  Intelligence  Officer  (S2), 

•  Assistant  S2  (AS2), 

•  Operations  and  Training  Officer  (S3), 

•  Assistant  S3  (AS3), 
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•  Supply  Officer  (S4), 

•  Fire  Support  Officer  (FSO),  and 

•  Air  Defense  Officer  (ADO). 

With  the  following  four  exceptions,  individuals  retained  the  same  battle  staff  position 
throughout  the  study: 

the  ABC  and  BC  exchanged  positions  on  one  day; 
the  S2  and  AS2  exchanged  positions  on  one  day; 

the  ABC  served  as  an  AS3  on  two  days  (the  ABC  position  remained  unfilled  on  those 
days);  and 

because  of  personal  problems,  the  individual  who  was  assigned  to  the  ADO  position 
was  present  only  for  a  part  of  the  first  day  of  the  study. 

The  reasons  for  the  changes  in  positions  are  not  completely  known.  The  move  of  the 
ABC  to  the  AS3  position  was  probably  motivated  by  the  need  to  increase  the  manpower 
available  for  mission  planning.  The  exchange  of  the  S2  and  AS2  positions  and  the  exchange  of 
the  BC  and  ABC  positions  were  probably  motivated  by  the  desire  to  provide  training  in  more 
than  one  position. 

Before  proceeding,  it  should  be  noted  that  the  BC  and  ABC  (when  the  ABC  position  was 
filled)  occupied  the  battalion  current  operations  cell  during  all  mission  execution  exercises  and 
most  of  the  time  between  mission  execution  exercises.  For  this  reason,  neither  the  BC  nor  the 
ABC  participated  actively  in  mission  planning.  It  should  also  be  noted  that  the  officers  who 
were  assigned  to  the  BC  and  ABC  positions  during  this  study  were  not  given  detailed 
instructions  on  the  duties  of  a  BC  or  an  ABC.  The  failure  to  instruct  the  BC  and  ABC  on  their 
specific  duties  and  responsibilities  probably  is  due  to  the  fact  that  (a)  the  position  of  BC  is  not  a 
Table  of  Organization  and  Equipment  (TO&E)  position  and  (b)  the  position  of  BC  is  not 
addressed  in  any  doctrinal  publication  (de  Oliveira,  1998). 

Research  Support  Staff 

Brigade  battle  staff.  The  brigade  TOC  was  used  solely  to  promote  exercise  realism  and 
to  control  the  exercises.  Hence,  all  personnel  who  served  as  members  of  the  brigade  battle  staff 
were  members  of  the  research  support  staff.  During  the  periods  when  missions  were  conducted, 
a  Commander,  an  S2,  an  S3,  an  S4,  and  an  FSO  usually  occupied  the  brigade  TOC.  The 
individuals  who  served  as  members  of  the  brigade  battle  staff  were  thoroughly  knowledgeable 
about  the  research  objectives  and  procedures  and  were  experienced  in  performing  the  duties  of 
their  assigned  staff  positions. 

Equipment  operators.  Members  of  the  research  support  staff  operated  all  of  the  digital 
systems  located  in  the  brigade  TOC.  In  addition,  members  of  the  research  support  staff  assisted 
the  battalion  battle  staff  in  operating  the  AS  AS  and  theMCS  during  mission  planning  and 
mission  execution.  Two  research  staff  members  occupied  the  battalion  plans  cell  during  all 
scheduled  mission  planning  sessions.  One  staff  member  provided  assistance  in  operating  the 
ASAS,  and  the  second  provided  assistance  in  operating  the  MCS.  Two  other  members  of  the 
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research  support  staff  occupied  the  battalion  TOC  current  operations  cell  during  all  mission 
execution  exercises  and  most  of  the  periods  between  mission  execution  exercises;  one  staff 
member  provided  assistance  in  operating  the  ASAS,  and  the  other  provided  assistance  in 
operating  the  MCS.  Two  staff  members  provided  parallel  support  in  the  brigade  TOC. 

Members  of  the  research  support  staff  were  responsible  for  programming  and  operating 
all  of  the  simulation  and  interface  components. 

Observers.  Ten  members  of  the  research  staff  served  as  observers.  They  were 
responsible  for  observing  and  recording  events  and  activities  that  occurred  during  all  mission 
planning  sessions  and  during  all  mission  execution  exercises.  One  observer  occupied  the  brigade 
TOC,  and  nine  observers  occupied  the  battalion  TOC.  The  brigade  TOC  observer  occupied  the 
current  operations  cell  at  all  times  when  members  of  the  research  staff  were  preparing  for  or 
conducting  a  mission.  The  locations  occupied  by  the  battalion  TOC  observers  are  described 
below. 


Five  of  the  battalion  TOC  observers  occupied  the  current  operations  cell  on  the  four  days 
during  which  missions  were  conducted.  Because  some  planning  activities  were  performed  in  the 
current  operations  cell,  observers  occupied  the  current  operations  cell  throughout  the  day.  The 
battalion  TOC  current  operations  cell  was  not  occupied  on  Day  3,  because  no  missions  were 
conducted  on  that  day;  all  members  of  the  battalion  battle  staff  were  engaged  in  mission  planning 
on  Day  3. 

Three  battalion  TOC  observers  occupied  the  plans  cell  or  the  adjacent  briefing  room 
during  all  scheduled  mission  planning  sessions  (including  mission  planning  briefings).  In  a  few 
instances,  battle  staff  members  engaged  in  mission  planning  activities  after  the  regularly 
scheduled  times.  No  observers  were  present  at  the  after-hours  mission  planning  sessions.  One 
observer  occupied  the  battalion  ALOC  during  all  times  when  battle  staff  members  were  present 
and  engaged  in  either  mission  planning  or  mission  execution  activities. 

Six  observers  were  Army  personnel  who  were  serving  active  duty  assignments  at  the 
USAAVNC;  two  were  Captains  and  four  were  NCOs.  The  two  Captains  were  qualified,  current 
Army  aviators.  The  remaining  four  observers  were  civilians;  three  were  government  employees, 
and  one  was  a  government  contractor.  Three  of  the  civilian  observers  were  actively  involved  in 
the  development  of  the  research  methodology,  including  the  development  of  (a)  the  observation 
procedures  and  forms  and  (b)  the  survey  instruments. 

Procedures 


Pre-Exercise  Training 

Prior  to  the  start  of  the  exercises,  it  was  necessary  to  train  the  members  of  the  battalion 
battle  staff,  company  commanders  and  pilots  who  occupied  the  FREDs,  and  the  observers.  The 
training  received  by  each  group  is  described  below. 
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Battle  staff  training.  All  members  of  the  battalion  battle  staff  received  four  hours  of 
training  on  the  layout  of  the  battalion  TOC  and  the  equipment  present  in  each  cell.  The  battle 
staff  members  were  given  a  comprehensive  briefing  on  (a)  the  functions  and  capabilities  of  each 
ATCCS  subsystem,  (b)  the  function  and  interpretation  of  the  information  displays,  and  (c)  the 
function  and  operation  of  the  radio  emulator.  Subsequently,  the  battle  staff  members  participated 
in  two  days  of  rehearsals  in  the  digitized  TOC.  The  rehearsals  provided  all  members  of  the  battle 
staff  an  opportunity  to  practice  performing  the  duties  of  their  assigned  positions  and  to  practice 
operating  the  digital  systems.  Members  of  the  research  staff  were  present  during  the  rehearsals 
to  help  battle  staff  members  learn  to  operate  the  digital  systems.  Battle  staff  member  training 
and  rehearsals  were  conducted  during  the  period  February  24-27,  1998. 

Company  commander  and  pilot  training.  The  company  commanders  and  pilots 
received  eight  hours  of  training  on  the  FREDs.  The  training  included  a  briefing  on  the  FRED 
displays  and  controls  and  the  characteristics  of  the  synthetic  battlespace  environment.  The 
company  commanders  and  pilots  spent  the  remaining  time  in  the  FREDs  acquiring  skills  in 
controlling  the  simulated  aircraft,  firing  weapons,  and  communicating.  Training  of  the  company 
commanders  and  pilots  was  conducted  on  February  21,  1998,  and  on  February  23,  1998. 

Observer  training.  During  a  90-minute  ATCCS  familiarization  briefing  on  February  24, 
1998,  observers  were  briefed  on  the  layout  of  the  battalion  TOC  and  the  function  of  each 
ATCCS  digital  subsystem  and  information  display.  The  following  day  (February  25,  1998),  all 
observers  attended  a  2-hr.  training  session  on  data  collection  procedures.  Observers  were  first 
briefed  on  the  problems  and  concerns  that  led  to  the  study  and  the  specific  objectives  of  the 
study.  Observers  were  briefed  on  (a)  the  data  collection  procedures  and  instruments  to  be 
employed,  '(b)  the  schedule  of  data  collection  activities,  and  (c)  the  specific  responsibilities  of 
each  observer.  Observers  then  were  provided  copies  of  the  observation  data  forms  (mission 
planning  form  and  mission  execution  form)  and  given  detailed  instruction  on  (a)  the  events  and 
activities  that  must  be  observed  and  (b)  the  manner  in  which  observation  data  must  be  recorded 
(encoded)  on  the  observation  data  forms. 

At  the  end  of  the  first,  second  and  final  days  of  observations,  all  observers  met  for  about 
1  hour  to  discuss  problems  that  were  encountered  during  the  day  and,  when  necessary,  to  refine 
the  observation  procedures  and  forms. 

Missions  Planning  and  Execution 

Members  of  the  battalion  battle  staff  were  engaged  in  mission  planning  sessions  and 
mission  execution  during  the  5-day  period  starting  at  0730  on  March  2  and  ending  at  1600  on 
March  6,  1998.  The  mission  planning  and  mission  execution  activities  were  a  part  of  a  one-week 
training  exercise  in  which  all  the  students  participated  at  the  end  of  the  AVOAC.  Because  the 
primary  purpose  of  the  exercises  was  to  train  the  AVOAC  students,  no  data  collection  methods 
or  procedures  could  be  used  that  might  degrade  the  training  effectiveness  of  the  exercises. 
Moreover,  it  was  not  possible  to  change  the  mission  scenarios  in  any  way  that  would  decrease 
the  training  effectiveness  of  the  exercises. 
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Described  below  is  the  scenario  that  provided  the  structure  for  mission  planning  and 
mission  execution  events  and  activities.  The  scenario  description  is  followed  by  a  description  of 
the  key  events  and  activities  that  occurred  each  day. 

Overview  of  scenario.6 7  The  study  scenario  was  based  on  the  Warfighter  Training 
Support  Package  (TSP)  for  the  Digitized  Aviation  Brigade  and  Division  Cavalry  Squadron.  The 
study  group  represented  the  1-4  Attack  Helicopter  Battalion  (AHB)  of  the  4th  Aviation  Brigade, 
4ID,  Mechanized.  In  addition  to  the  three  companies  of  AH-64As  (24  aircraft),  a  platoon  of  UH- 
60s  (4  aircraft)  and  a  team  of  OH-58Ds  (2  aircraft)  were  provided  by  OPCON  (operational 
control)  to  the  AHB.  The  brigade  level  OPORD  and  fragmentary  orders  (FRAGOs)  issued  to  the 
AHB  were  developed  by  the  Warfighting  Futures  Division  (WFD)  of  DOTDS,  based  on  the  TSP 
division  level  order  and  FRAGOs  1-3.  A  “91 1”  mission  also  was  issued  on  the  last  day  of  the 
study  based  on  TSP  FRAGO  4.  The  term  "911"  is  an  informal  descriptor  for  any  emergency  or 
unplanned  set  of  events  that  necessitate  a  hasty  operation.  Additional  complementary  missions 
(e.g.,  combat  search  and  rescue  [CSAR],  reconnaissance,  general  support  [GS])  were  executed 
on  order  from  the  brigade  TOC  (control  cell). 

For  this  study,  each  of  the  FREDs  that  was  configured  as  an  AH-64A  represented  a 
company  of  eight  AH-64A  Apache  helicopters.  One  FRED  configured  as  a  UH-60  Blackhawk 
helicopter  represented  a  platoon  of  four  UH-60s.  Two  FREDs  configured  as  OH-58D  Kiowa 
Warrior  helicopters  represented  a  team  of  two  OH-58Ds. 

Day  1  events/activities.  The  base  OPORD  was  issued  at  the  start  of  Day  1 .  Members 
of  the  battalion  battle  staff  commenced  planning  immediately  after  they  received  the  OPORD 
and  continued  planning  throughout  the  day.  Other  events  that  occurred  during  Day  1  are  listed 
below. 

•  Brigade  sent  an  urgent  resupply  request  to  the  battalion. 

•  The  resupply  mission  was  assigned  to  the  UH-60  platoon. 

•  During  the  resupply  mission,  the  UH-60  crew  observed  and  reported  enemy 
reconnaissance  vehicles. 

•  Brigade  ordered  the  OH-58D  team  to  gain  contact  with  enemy  vehicles  and  to 
develop  the  situation. 

•  The  TUAV  supported  the  missions  and  supported  a  search  for  downed  aircraft. 

•  Company  commanders  were  briefed  on  the  mission  scheduled  for  Day  2. 

Day  2  events/activities.  The  main  mission  during  Day  2  was  to  conduct  a  deep  attack 
against  an  enemy  surface-to-surface  missile  site.  The  activities  commenced  with  the  launch  of 
the  TUAV.  Shortly  thereafter,  the  brigade  commander  issued  a  FRAGO  for  an  AH-64  company 
to  conduct  a  screen  mission.  Subsequently,  the  brigade  commander  issued  a  FRAGO  to  conduct 
a  deep  attack  against  the  missile  site. 


6  The  description  of  the  study  scenario  was  paraphrased  from  the  description  in  die  Exercise  Support  Plan  prepared 
by  Coleman  Research  Corporation  (Schmidt  and  Shehan,  1998). 

7  The  date  corresponding  widi  Day  1  is  March  2,  1998. 
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Day  3  everits/activities.  Day  3  was  devoted  solely  to  mission  planning.  No  missions 
were  executed  in  the  battalion  TOC  on  Day  3. 

Day  4  events/activities.  Day  4  activities  commenced  with  a  rehearsal  of  the  deep  attack 
mission  scheduled  for  the  afternoon  of  Day  4.  The  mission  rehearsal  was  completed,  and  the 
deep  attack  mission  was  conducted  as  scheduled.  The  rehearsal  and  the  mission  involved  the  use 
of  the  TUAV  for  target  and  route  reconnaissance  and  the  use  of  the  OH-58Ds  for  reconnaissance 
and  attack  support. 

Dav  5  events/activities.  Day  5  activities  commenced  with  a  rehearsal  of  the  hasty  attack 
mission  scheduled  for  the  afternoon  of  Day  5.  The  mission  rehearsal  was  completed 
successfully,  and  the  deep  attack  mission  was  conducted  as  scheduled.  The  rehearsal  and  the 
mission  involved  the  use  of  the  TUAV  for  target  and  route  reconnaissance  and  the  use  of  the 
OH-58Ds  for  reconnaissance  and  attack  support. 

Demographic  Questionnaire  and  Consent  Form 

All  members  of  the  battalion  battle  staff  reviewed  and  signed  a  Consent  Agreement.  The 
Consent  Agreement  described  the  purpose  of  the  study  and  the  type  of  information  that  the  battle 
staff  members  would  be  asked  to  disclose.  The  Consent  Agreement  explained  that  the 
information  would  remain  confidential  and  that  battle  staff  members  had  the  right  to  refuse  to 
provide  information  without  risk  of  negative  consequences. 

At  the  end  of  Day  1,  all  members  of  the  battalion  battle  staff  completed  a  Demographic 
Questionnaire.  The  Demographic  Questionnaire  was  designed  to  acquire  information  about  staff 
members’  age,  grade,  Specialty  Skill  Indicator  (SSI),  Primary  Occupational  Specialty  (PMOS), 
years  of  military  experience,  current  duty  assignment,  education  courses  and  technical  schools 
attended,  experience  as  . a  battle  staff  member,  and  knowledge  of  the  functional  capabilities  and 
operation  of  the  digital  systems  present  in  the  battalion  ATCCS. 

Appendix  A  contains  a  copy  of  the  Consent  Agreement  and  a  copy  of  the  Demographic 
Questionnaire. 

Mission  Planning  Questionnaire 

A  Mission  Planning  Questionnaire  was  completed  by  all  members  of  the  battalion  battle 
staff  on  three  occasions.  On  the  first  occasion,  battle  staff  members  answered  questions  about 
the  planning  that  was  accomplished  on  Day  1  (March  2)  and  on  part  of  Day  2  (March  3).  On  the 
second  occasion,  battle  staff  members  answered  questions  about  the  planning  that  was 
accomplished  on  Day  3  (March  4).  On  the  third  occasion,  battle  staff  members  answered 
questions  about  the  planning  that  was  conducted  on  Day  4  (March  5).  Appendix  B  contains  a 
copy  of  the  Mission  Planning  Questionnaire  and  the  list  of  MDMP  steps  and  substeps  that  was 
attached  to  the  questionnaire;  a  brief  description  of  the  questionnaire  items  is  presented  below. 

The  first  group  of  items  on  the  Mission  Planning  Questionnaire  asked  battle  staff 
members  to  provide  information  about  the  amount  of  time  they  spent  planning  the  mission  and 
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the  difficulty  of  planning  the  mission  (relative  to  the  difficulty  of  planning  a  live  operational 
mission).  The  following  item  asked  battle  staff  members  to  rate  mission  realism  relative  to  the 
realism  of  a  live  operational  mission.  One  item  asked  battle  staff  members  to  indicate  the  need 
for  and  the  functions  of  an  information  manager  who  would  manage  information  during  mission 
planning.  A  series  of  items  asked  battle  staff  members  to  provide  a  separate  response  for  each  of 
six  MDMP  steps.  Responses  for  each  MDMP  step  were  required  for  each  of  the  following 
queries: 

•  highest  workload  experienced, 

•  adequacy  of  time  and  information  (type  and  accessibility), 

•  errors  or  oversights  made, 

•  adequacy  of  knowledge  of  the  digital  equipment  operation  and  functions, 

•  adequacy  of  TTPs, 

•  need  for  additional  automation,  and 

•  adequacy  of  prior  training. 

Every  item  had  an  open-end  part  that  invited  the  battle  staff  member  to  provide  a  written 
explanation  of  the  answers  or  opinions  expressed  in  the  quick-response  (i.e.,  checklist)  part  of 
the  item. 


Mission  Planning  Observations 


Three  observers  were  responsible  for  observing  selected  events  and  activities  that 
occurred  during  the  mission  planning  sessions.  The  three  observers  worked  as  a  team  in 
completing  the  Mission  Planning  Observation  Form  shown  in  Appendix  C.  The  mission 
planning  observers  were  responsible  for  recording  selected  information  about  each  of  the 
following  MDMP  steps. 


•  Step  1 :  Receive  the  Mission 

•  Step  2:  Analyze  the  Mission 

•  Step  3:  Develop  the  Course  of  Action  (CO  A) 

•  Step  4:  Analyze  the  COA 

•  Step  5:  Compare  the  COA 

•  Step  6:  Approve  the  COA 

Observers  recorded  the  time  that  work  on  each  MDMP  step  commenced  and  the  time  that 
work  on  the  step  was  completed.  As  each  MDMP  step  was  being  completed,  observers  recorded 
the  MDMP  substeps  that  were  accomplished,  the  battle  staff  members  who  participated  in  each 
substep,  and  the  members  of  the  support  staff  who  participated  in  the  step.  When  an  MDMP  step 
had  been  completed,  observers  were  instructed  to  identify  problems  encountered  in  performing 
the  MDMP  step  and  the  battle  staff  member(s)  who  encountered  the  problem.  Space  was 
provided  on  the  Mission  Planning  Observation  Form  for  observers  to  describe  instances  of  each 
of  the  following  problems: 


•  important  information  not  available, 

•  important  information  not  easily  accessible, 
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•  excessive  workload, 

•  lack  of  knowledge  about  the  functional  capabilities  of  digital  equipment, 

•  lack  of  skill  in  operating  digital  equipment, 

•  inadequate  TTPs, 

•  lack  of  training/experience  on  the  MDMP,  and 

•  other  problems. 

After  the  mission  planning  had  been  completed,  observers  were  required  to  provide 
written  descriptions  of  mission  planning  errors  or  oversights  that  they  observed.  Observers  also 
were  asked  to  record  their  judgments  about  (a)  the  need  for  increased  automation,  (b)  the  need 
for  an  information  manager,  and  (c)  the  need  for  additional  training  of  battle  staff  members. 

Mission  Execution  Observations 

Six  observers  occupied  the  battalion  TOC  for  the  entirety  of  Day  1,  Day  2,  Day  4,  and 
Day  5.  One  observer  was  assigned  to  each  of  five  observation  stations  within  the  battalion  TOC 
and  was  instructed  to  observe  the  activities  of  the  members  of  the  battle  staff  who  were  present  at 
that  station.  The  sixth  observer,  a  backup  observer,  occupied  an  observation  station  during  the 
time  that  the  regular  observer  took  a  break.  The  five  observation  stations  are  listed  below  along 
with  the  personnel  who  were  present  at  the  observation  station  all  or  some  of  the  time.  Station  5 
was  located  in  the  ALOC;  all  other  stations  were  located  in  the  current  operations  cell. 

•  Station  1 :  S2,  AS2  (AS  AS  operator),  support  staff  member  (AS  AS  operator),  and  BC 

•  Station  2:  S3,  AS3  (MCS  operator),  support  staff  member  (MCS  operator),  and  ABC 

•  Station  3 :  CO  and  XO 

•  Station  4:  FSO  and  support  staff  member  (AFATDS  operator) 

•  Station  5:  S  I,  S4,  support  staff  member  (CSSCS  operator) 

Appendix  D  shows  a  copy  of  the  Mission  Execution  Observation  Form8  that  observers 
used  to  record  information  about  events  and  activities  that  occurred  during  the  execution  of  a 
mission.  Observers  were  instructed  to  record  information  on  (a)  voice  communication  events 
and  (b)  other  problems  and  events.  For  each  voice  communication  at  an  observer’s  station,  the 
observer  recorded  the  time  the  voice  communication  was  initiated,  the  person  who  initiated  the 
communication,  and  the  person  to  whom  the  communication  was  directed.  Observers  also 
recorded  the  content  of  the  communication  (actual  or  paraphrased)  and  classified  the  message  by 
checking  one  of  five  alternatives,  which  are  listed  and  defined  below. 

•  Request  for  Information:  This  alternative  was  checked  when  the  message  was  a 
request  for  information. 

•  Response  to  Request:  This  alternative  was  checked  when  the  message  was  a  response 
to  a  prior  request  for  information. 


8  The  Mission  Execution  Observation  Form  shown  in  Appendix  D  is  a  smaller  version  of  the  form  that  was  used  by 
observers.  Although  the  Form  in  Appendix  D  shows  all  the  information  that  appeared  on  the  actual  form,  the  space 
available  for  notes  was  reduced  in  size  to  make  the  form  fit  on  a  standard  ZVi-in  x  1 1  -in  sheet. 
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•  Information  Push:  This  alternative  was  checked  when  an  individual  conveyed 
information  to  another  individual  without  being  requested  to  do  so. 

•  Situation  Update:  This  alternative  was  checked  when  the  communication  was  a 
situation  update  that  was  provided  by  one  individual  for  the  benefit  of  other 
individuals  (two  or  more). 

•  Coordination:  This  alternative  was  checked  when  the  purpose  of  the  communication 
event  was  an  extended  dialog  intended  to  coordinate  the  actions  of  the  two  or  more 
persons  involved  in  the  conversation. 

The  observers  were  instructed  to  record  (a)  problems  that  may  have  influenced  the 
outcome  of  the  mission,  (b)  key  decision  points  that  occurred  during  the  mission,  and  (c)  the  time 
at  which  the  problem  or  decision  point  occurred.  The  Mission  Execution  Observation  Form  was 
designed  such  that  observers  could  place  a  checkmark  in  a  box  that  corresponded  with  events  of 
particular  interest.  The  events  that  could  be  checked  by  observers  are  listed  and  defined  below. 

•  Excessive  Workload:  This  alternative  was  checked  when  the  observer  judged  that  the 
workload  of  a  staff  member  was  excessive. 

•  Inadequate  Situation  Awareness:  This  alternative  was  checked  when  the  observer 
judged  that  a  staff  member  lacked  an  adequate  level  of  situation  awareness. 

•  Decision  Point:  This  alternative  was  checked  when  the  situation  required  a  staff 
member  to  make  a  decision,  whether  or  not  a  decision  was  made. 

•  Decision  Error:  This  alternative  was  checked  when  a  decision  by  a  staff  member  may 
have  had  an  adverse  effect  on  the  mission  outcome. 

•  Skill  Deficiency:  This  alternative  was  checked  when  there  was  evidence  that  a  staff 
member  lacked  the  knowledge  or  skill  needed  to  perform  his  or  her  job. 

•  Other:  This  alternative  was  checked  when  an  event  other  than  those  listed  above  may 
have  had  an  adverse  effect  on  the  mission  outcome. 

When  any  of  the  above  options  were  checked,  observers  recorded  the  position  of  the  staff 
member  who  experienced  the  event  (problem  or  decision  point)  and  wrote  a  brief  description  of 
the  event. 

After  Action  Reviews  ( AARsI 

An  AAR  was  conducted  immediately  after  the  completion  of  each  mission  execution 
exercise.  All  members  of  the  battalion  battle  staff,  the  company  commanders,  the  FRED  pilots, 
and  most  members  of  the  research  support  staff  attended  the  AARs.  Each  AAR  commenced 
with  a  detailed  review  and  assessment  of  the  mission  just  completed.  Although  a  member  of 
either  the  research  support  staff  or  instructional  staff  directed  the  review  and  assessment, 
members  of  the  battalion  battle  staff  were  often  invited  to  comment  on  key  events  that  occurred 
and  key  problems  that  were  encountered  during  the  mission. 

Following  the  review  and  assessment  of  the  mission,  all  attendees  were  invited  to  identify 
and  discuss  problems  encountered  and  “lessons  learned”  from  either  the  planning  or  the 
execution  of  the  mission. 


25 


Trainine  Effectiveness  Questionnaire 


After  completing  the  last  mission  execution  exercise  (Day  5),  all  members  of  the 
battalion  battle  staff  completed  a  Training  Effectiveness  Questionnaire  (see  Appendix  E).  The 
Training  Effectiveness  Questionnaire  required  battle  staff  members  to  rate  their  level  of 
agreement  with  30  different  statements  related  to  the  training  effectiveness  of  the  ATCCS  suite 
of  digital  systems  and  virtual  systems. 

Post-Exercise  Interviews 

During  the  week  following  the  completion  of  the  exercises,  two  members  of  the  research 
staff  conducted  detailed  interviews  with  the  XO,  the  SI,  the  S2,  the  S3,  and  the  S4.  Three 
interview  sessions  were  conducted:  one  with  the  XO,  one  with  the  SI  and  S4,  and  one  with  the 
S2  and  S3. 

The  interview  with  the  XO  focused  on  two  topics.  The  first  topic  was  the  adequacy  of 
the  ATCCS  digital  systems  for  establishing  and  sustaining  the  situation  awareness  of  the 
commander.  The  second  topic  was  the  need  for  an  information  manager  and  the  duties  of  an 
information  manager. 

The  interview  with  the  S2  and  S3  focused  on  the  utility  of  the  digital  systems  for  mission 
planning.  The  S2  and  S3  were  questioned  in  detail  about  the  MDMP  tasks  that  were  facilitated 
by  the  digital  systems  and  those  that  were  not  facilitated.  The  S2  and  S3  were  invited  to  identify 
and  discuss  design  improvements  that  may  increase  the  utility  of  the  digital  systems  for  mission 
planning. 

The  interview  with  the  SI  and  S4  focused  on  the  utility  of  the  CSSCS  for  mission 
planning  and  mission  execution.  The  SI  and  S4  were  asked  to  identify  the  types  and  causes  of 
the  problems  they  encountered  during  mission  planning  and  execution.  They  also  were  asked  to 
identify  and  discuss  design  improvements  that  may  increase  the  utility  of  the  CSSCS  for  mission 
planning  and/or  mission  execution. 


Findings 

This  subsection  has  four  major  parts.  The  first  part  describes  the  demographic 
characteristics  of  the  battle  staff  members  who  served  as  members  of  the  battalion  battle  staff. 
The  second  part  describes  the  data  compiled  from  questionnaires  about  mission  planning  and 
from  direct  observations  of  each  mission  planning  session.  The  third  part  describes  the  data 
compiled  from  direct  observations  of  the  execution  of  each  mission  and  from  survey  forms  that 
were  completed  by  battle  staff  members  after  the  completion  of  each  mission.  The  fourth  part  is 
a  synopsis  of  the  problems  described  by  battle  staff  members.  The  synopsis  includes  problems 
described  by  battle  staff  members  on  their  survey  forms  and  problems  discussed  during  the 
AARs  or  the  post-exercise  interviews. 
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Data  Presentation  and  Analysis 


As  was  stated  earlier,  this  was  an  observational  study  rather  than  a  classical  experiment. 
The  intent  was  to  observe  the  manner  in  which  the  battle  staff  members  performed  their  jobs 
(individually  and  collectively)  and  to  draw  inferences  about  the  manner  in  which  the  behaviors 
influenced  the  outcomes.  Accordingly,  inferential  statistical  procedures  are  inappropriate. 
Descriptive  statistics  were  employed  in  documenting  the  findings.  The  descriptive  statistics 
presented  in  the  following  pages  include  percentage  values,  frequencies,  means,  medians,  and 
modes.  However,  some  readers  may  desire  an  estimate  of  the  likelihood  that  a  difference  among 
percentage  values  (or  differences  among  frequencies)  could  have  occurred  by  chance.  To  satisfy 
this  need,  Chi  Square  (symbolized  x2)  values  have  been  calculated  and  presented  in  a  number  of 
instances  (Guilford  and  Fructer,  1978).  These  provide  a  basis  for  evaluating  the  likelihood  that  a 
difference  between  frequencies  represents  a  chance  fluctuation.  However,  it  is  important  to 
emphasize  that  the  x2  values  cannot  be  used  to  draw  inferences  about  whether  differences  are 
representative  of  the  differences  that  would  occur  with  other  battle  staffs  or  other  missions.  In 
other  words;  the  reader  is  cautioned  not  to  generalize  the  results  indicated  by  the  %2  values 
presented  to  any  other  exercise  or  other  sample  of  behavior.  Because  of  the  type  and  amount  of 
data  compiled  during  this  study,  there  are  virtually  thousands  of  differences  in  frequencies  (or 
percentage  values)  that  could  be  assessed.  Not  all  of  the  possible  comparisons  have  relevance  to 
the  objectives  outlined  for  the  study. 

In  some  instances,  only  two  frequencies  were  compared.  For  these,  the  null  hypothesis 
being  tested  is  that  the  two  frequencies  are  the  same  (e.g.,  the  number  of  battle  staff  members 
who  responded  “yes”  to  a  questionnaire  item  is  the  same  as  the  number  who  responded  “no”).  In 
other  instances,  a  contingency  %2  value  was  computed  to  test  the  null  hypothesis  that  more  than 
two  frequencies  are  the  same  (e.g.,  the  number  of  battle  staff  members  who  responded  “yes”  was 
the  same  for  each  of  the  seven  MDMP  steps).  Some  comparisons  involve  samples  that  are  not 
independent.  For  example,  participants  were  given  the  same  set  of  surveys  after  each  mission. 
Their  responses  would  be  considered  repeated  measures  and  therefore  not  independent.  In  these 
cases  the  sample  values  have  been  converted  to  differences  between  or  among  samples  matched 
by  respondent.  The  null  hypothesis  tested  is  that  the  difference  scores  equal  zero.  Unless  stated 
otherwise,  %2  statistics  are  not  be  considered  significant  indicators  of  response  differences  unless 
the  probability  that  the  %2  value  occurred  by  chance  equals  or  is  less  than  5%  (p<  .05). 

Battle  Staff  Member  Characteristics 

Some  relevant  characteristics  of  the  individuals  who  served  as  members  of  the  battalion 
battle  staff  are  summarized  in  Table  1.  The  information  in  Table  1  was  extracted  from  the 
Demographic  Questionnaire  (see  Appendix  A)  that  battle  staff  members  completed  on  the  first 
day  of  the  study.  The  first  row  of  Table  1  shows  the  battle  staff  position  to  which  each 
individual  was  assigned  at  the  outset  of  the  study.  As  stated  earlier,  most  battle  staff  members 
remained  in  the  position  to  which  they  were  assigned  initially.  On  one  day,  the  S2  exchanged 
position  with  the  AS2  and  the  BC  exchanged  positions  with  the  ABC.  On  two  days,  the  ABC 
assumed  the  position  of  BC,  and  the  BC  served  as  another  AS3;  the  position  of  ABC  was  not 
filled  on  these  days.  Because  the  position  of  ADO  was  filled  for  only  part  of  one  day,  no 
information  is  presented  for  the  individual  who  was  assigned  to  the  ADO  position. 
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All  battle  staff  members  were  rated  rotary  wing  aviators  and  commissioned  officers.  The 
CO  and  XO  held  the  rank  of  Major  (MAJ)  and  had  17  and  15  total  years  of  military  service, 
respectively.  Ail  of  the  remaining  battle  staff  members  held  the  rank  of  Captain  (CPT)  and  had 
between  5  and  1 1  total  years  of  military  service.  All  battle  staff  members  except  the  CO  were  in 
the  process  of  completing  the  AVO  AC  at  the  time  the  ATCCS  study  was  conducted;  the  XO  was 
the  class  leader  of  AVOAC  class  98-1. 

The  CO,  XO,  and  S3  had  served  as  a  company  commander,  and  the  CO  and  XO  had 
served  as  a  battalion  assistant  S3.  The  XO  also  had  served  as  a  battalion  SI.  Five  of  the 
remaining  8  battle  staff  members  (S2,  AS3,  S4,  FSO,  and  ABC)  had  at  least  5-months  of 
experience  in  a  battalion  battle  staff  position.  However,  none  of  the  1 1  battle  staff  members  had 
prior  experience  in  the  battle  staff  position  to  which  they  were  assigned  in  the  ATCCS  study. 
Three  battle  staff  members  (SI,  AS2,  and  BC)  had  no  prior  experience  serving  as  a  member  of  a 
battalion  battle  staff. 


Table  1 

Characteristics  of  the  individuals  who  served  as  a  member  of  the  battalion  battle  staff. 


QUESTIONNAIRE 

ITEM 

Assigned  Battle 
Staff  Position 


Current 

Rank 


Age 

(years) 


Active  Component 
Service  (years) 


Reserve  Component 
Service  (years) 


Total  Service 
(years) 


Experience  as  Battle  Staff 
Member  (months) 


.S 

n 


BATTLE  STAFF  MEMBER  RESPONSES 
CO  XO  SI  S2  AS2  S3  AS3  S4  FSO  BC  ABC 


MAJ  MAJ  CPT  CPT  CPT  CPT  CPT  CPT  CPT  CPT  CPT 

35  33  29  29  28  28  31  29  26  28  27 


15  10 


6 


0  0 


S4 

FSO 

CPT 

CPT 

29 

26 

11 

5 

0 

0 

17  15 


48  24 


0  13  0  15 


2/1  2/1  3/1  3/2 


2/1  2/1  2/2 


iani 


l/l 


l/i  l/l 


2/1  1/1  2/1  1/1 


2/1  1/1  2/1  1/1 


0  20 


2/2  2/1 


2/2  2/1 


2/1  1/1  2/1  1/1  2/2  3/2  1/1  1/1  1/1  2/2  2/2 


Knowledge  of  ASAS 
Capabilities/Operation 


Knowledge  of  AFATDS  . 
Capabilities/Operation 


Knowledge  of  MCS 
Capabilities/Operation  J 


Knowledge  of  FAADEO 
Capabilities/Operation 


Note.  The  pairs  of  numbers  in  the  last  five  rows  are  rating  values  used  by  battle  staff  members  to  rate  their 
knowledge  of  the  functional  capabilities  of  the  corresponding  digital  system  (first  number)  and  their  skill  in 
operating  the  system  (second  number).  The  rating  values  varied  from  1  (no  knowledge/skill)  through  5  (high 
knowledge/skill). 


2/2  3/3 


2/2  1/1 
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Battle  staff  members  were  asked  to  rate  (a)  their  knowledge  of  the  functional  capabilities 
of  each  of  five  digital  systems  and  (b)  their  skill  in  operating  each  system.  The  rating  scale  used 
by  battle  staff  members  varied  from  none  (rating  of  1)  through  moderate  (rating  of  3)  to  high 
(rating  of  5).  Battle  staff  members’  ratings  are  shown  in  the  five  lines  at  the  bottom  of  Table  1. 
Both  ratings  are  shown  in  the  same  cell;  the  first  number  (before  the  slash)  is  the  rating  for 
knowledge  of  functional  capabilities,  and  the  second  number  (after  the  slash)  is  the  rating  for 
skill  in  operating  the  system. 

The  ratings  show  that  few  battle  staff  members  had  even  a  moderate  (rating  of  3)  level  of 
knowledge  about  the  functional  capabilities  of  any  digital  system;  even  fewer  battle  staff 
members  had  even  a  moderate  level  of  skill  in  operating  any  system.  Of  the  55  ratings,  43  (78%) 
are  1  (none)  or  2  (slight);  the  remaining  12  ratings  (22%)  are  3  (moderate).  These  ratings  are 
consistent  with  numerous  comments  by  battle  staff  members  that,  at  the  outset  of  the  study,  they 
were  severely  handicapped  by  their  knowledge  and  skill  deficiencies  in  using  the  digital  systems 
for  mission  planning  and  for  mission  execution. 

Mission  Planning 

Data  about  mission  planning  came  from  (a)  a  mission  planning  questionnaire  completed 
by  each  battle  staff  member  and  (b)  notes  and  data  forms  completed  by  the  members  of  the 
research  support  staff  who  were  responsible  for  observing  all  mission  planning  activities.  The 
data  from  these  two  sources  are  discussed  in  turn. 

Mission  Plannin£  Questionnaire  Responses 

Each  battle  staff  member  completed  a  mission  planning  questionnaire  (see  Appendix  B) 
on  three  occasions.  On  the  first  occasion,  battle  staff  members  answered  questions  about  the 
planning  that  was  accomplished  on  Day  1  (March  2)  and  a  part  of  Day  2  (March  3).  On  the 
second  occasion,  battle  staff  members  answered  questions  about  the  planning  that  was 
accomplished  on  Day  3  (March  4).  On  the  third  occasion,  battle  staff  members  answered 
questions  about  the  planning  that  was  conducted  on  Day  4  (March  5).  Throughout  the  discussion 
of  mission  planning,  it  is  implied  that  a  separate  mission  was  planned  during  each  of  the 
planning  sessions.  For  example,  the  tables  and  charts  show  planning  data  for  Mission  1,  Mission 
2,  and  Mission  3.  The  planning  conducted  during  the  sessions  dealt  with  different  FRAGOs,  but 
the  OPORD  remained  the  same.  Although  all  of  the  planning  was  in  response  to  the  same 
OPORD,  the  first  and  the  third  mission  planning  sessions  required  battle  staff  members  to 
perform  all  of  the  MDMP  steps,  and  the  second  session  required  battle  staff  members  to  perform 
most  of  the  MDMP  steps. 

Because  few  battle  staff  members  changed  battle  staff  positions  from  one  planning 
session  to  the  next,  all  responses  are  presented  by  battle  staff  position  rather  than  by  individual. 

Time  spent  planning.  The  BC  and  ABC  were  not  required  to  play  an  active  role  in 
mission  planning.  The  BC  spent  no  time  on  mission  planning,  and  the  ABC  spent  only  3  hours 
on  planning  Mission  2  and  1  hour  helping  to  plan  Mission  3.  The  hours  the  remaining  battle 
staff  members  spent  planning  each  mission  are  shown  in  Table  2.  For  Mission  2  and  3,  two  • 
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AS3s  participated  in  mission  planning.  Table  2  shows  the  planning  time  each  of  the  AS3s  spent 
planning  Mission  2  and  Mission  3  (e.g.,  the  entry  14/10  means  one  AS3  spent  14  hours,  and  the 
second  AS3  spent  10  hours  planning  Mission  3). 

The  SI  spent  only  2.0  hours  planning  each  of  the  three  missions.  All  other  battle  staff 
members  spent  no  less  than  6.0  hours  and  as  many  as  16  hours  planning  a  mission.  When 
averaged  across  battle  staff  members,  the  mean  time  per  mission  ranges  from  9.6  hours  (Mission 
1)  to  10.8  hours  (Mission  3).  When  averaged  across  missions,  the  mean  time  per  mission  is 
similar  for  all  battle  staff  members  except  the  SI.  The  mean  planning  time  for  the  SI  is  only  2.0 
hours;  whereas,  the  mean  planning  time  for  the  remaining  battle  staff  members  ranges  from  9.7 
hours  (S3)  to  13.3  hours  (FSO). 

Table  2 

Self-reports  of  time  (in  hours)  spent  by  battle  staff  members  planning  each  of  three  missions. 


BATTLE  STAFF  MEMBER 

MISSION 

CO 

xo 

SI 

S2 

AS2 

S3 

AS3 

S4 

FSO 

Mean 

1 

10 

10 

2 

10 

12 

10 

10 

12 

10 

9.6 

2 

10 

15 

2 

15 

9 

11 

10/10 

6 

16 

10.4 

3 

12 

12 

2 

8 

16 

8 

14/10 

12 

14 

10.8 

Mean 

10.7 

12.3 

2.0 

11.0 

12.3 

9.7 

10.8 

10.0 

13.3 

10.3 

These  findings  leave  no  doubt  that  a  substantial  amount  of  time  was  spent  planning  each 
of  the  three  missions.  The  small  variability  in  mean  planning  time  from  one  mission  to  another 
may  indicate  that  the  three  missions  were  about  equal  in  complexity  and  planning  difficulty.9 
However,  it  is  probable  that  the  tendency  to  use  all  the  planning  time  available  also  contributed 
to  the  small  variability  in  planning  time  across  missions. 

Before  proceeding,  it  is  important  to  note  that  self-reports  of  time  spent  planning  cannot 
be  treated  as  a  reliable  indicator  of  battle  staff  members’  relative  workload  during  planning. 
Battle  staff  members  tended  to  report  the  number  of  hours  that  were  required  to  complete  an 
MDMP  step10  rather  than  the  number  of  hours  they  were  actively  engaged  in  the  tasks  that  were 
required  to  complete  the  MDMP  step.  For  this  reason,  planning  time  is  less  useful  for  assessing 
the  relative  workload  of  the  battle  staff  members  than  other  indicators  such  as  (a)  battle  staff 
members’  workload  ratings  and  (b)  the  planning  task  responsibilities  of  different  battle  staff 
members.  Both  workload  ratings  and  planning  task  responsibilities  are  discussed  in  a  later 
subsection. 


9  This  interpretation  is  supported  by  battle  staff  members’  ratings  of  the  difficulty  of  planning  each  mission.  Battle 
staff  members’  difficulty  ratings  are  discussed  in  the  next  subsection  of  the  report. 

10  The  MDMP  specifies  planning  “steps”  and  “substeps”  that  must  be  accomplished  [see  FM  101-5  (Army,  1997)] 
when  planning  a  mission.  Hereafter,  the  term  “task”  will  be  used  when  referring  to  general  planning  activities.  •  The 
terms  “steps”  and  “substeps”  will  be  used  when  discussing  the  planning  activities  that  were  specified  by  the  MDMP. 
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Planning  difficulty.  Battle  staff  members  were' asked  to  rate  the  difficulty  of  planning 
each  of  the  three  missions  relative  to  planning  an  operational  mission  of  the  same  type  (Item  2). 
Responses  to  the  5 -point  rating  scale  range  from  much  easier  (rating  of  1)  through  about  the 
same  (rating  of  3)  to  much  more  difficult  (rating  of  5).  The  difficulty  ratings  of  each  battle  staff 
member  who  participated  in  mission  planning  are  shown  in  Table  3. 11  As  stated  earlier,  two 
AS3s  participated  in  the  planning  of  Mission  2  and  Mission  3.  Table  3  shows  difficulty  ratings 
for  both  AS3s;  the  ratings  of  the  two  AS3s  are  separated  by  a  slash  (/). 

Table  3 


Battle  staff  members’  ratings  of  the  difficulty  of  mission  planning. 


BATTLE  STAFF  MEMBER 

MISSION 

CO 

XO 

SI 

S2 

AS2 

S3 

AS3 

S4 

FSO 

Mean 

4 

4 

2 

5 

4 

4 

4 

4 

4 

2  (March  5) 

3 

4 

3 

5 

2 

4 

2 

- 

3 

4 

2 

5 

3 

4 

4 

3 

mm 

:Tlie  two  numbers  are  ratings  by  the  two  individuals  who  served  as  an  AS3  during  the  planning  of  the 
corresponding  mission. 


The  distribution  of  ratings  shows  that,  as  a  group,  battle  staff  members  judged  that  the 
missions  were  substantially  more  difficult  to  plan  than  comparable  operational  missions. 
Specifically,  53.6%  of  the  ratings  are  4  (somewhat  more  difficult),  and  another  10.7%  are  5 
(much  more  difficult).  Together,  ratings  of  2  (somewhat  easier)  and  3  (about  the  same)  account 
for  only  35.8%  of  all  difficulty  ratings.  The  mean  ratings  shown  in  Table  3  indicate  that,  as  a 
group,  battle  staff  members  judged  that  the  difficulty  of  the  three  missions  was  about  the  same. 
Although  the  mean  rating  for  Mission  1  is  slightly  larger  than  the  mean  ratings  for  Mission  2  and 
Mission  3,  only  the  CO  and  the  AS2  indicated  that  Mission  1  was  more  difficult  to  plan  than  the 
other  two  missions.  More  importantly,  the  battle  staff  members  who  were  most  actively 
involved  in  mission  planning  (XO,  S2,  and  S3)  rated  mission  planning  difficulty  the  same  for  all 
three  missions.  These  data  provide  no  evidence  that  (a)  mission  planning  difficulty  differed 
substantially  from  one  mission  to  another  or  (b)  mission  planning  difficulty  declined 
substantially  with  practice. 

Information  about  the  reasons  for  planning  difficulties  was  provided  by  responses  to  Item 
3,  which  asked  battle  staff  members  to  Describe  ways  in  which  the  planning  of  today’s  mission 
was  more  difficult  and  the  ways  it  was  l_ess  difficult  than  the  planning  of  an  operational  mission 
of  the  same  type.  Two  reasons  for  planning  difficulty  were  cited  by  most  of  the  battle  staff 
members  who  were  actively  involved  in  mission  planning. 

First,  six  of  the  nine  battle  staff  members  who  were  actively  involved  in  mission  planning 
cited  “knowledge  and  skill  deficiencies  in  using  digital  equipment”  as  a  reason  for  mission 


1 1  The  BC  and  ABC  occupied  the  current  operations  cell  when  other  members  of  the  battle  staff  were  engaged  in 
mission  planning.  Hence,  the  battle  staff  members  who  served  as  BC  and  ABC  did  not  participate  actively  in  ' 
mission  planning. 
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planning  difficulty.  The  FSO,  the  SI,  and  the  S4  did  not  cite  knowledge/skill  deficiencies  as  a 
reason  for  mission  planning  difficulties.  The  most  probable  reason  that  knowledge  and  skill 
deficiencies  in  using  digital  equipment  was  not  cited  by  the  FSO  is  that  the  AFATDS  was  not 
operational  during  the  planning  of  any  mission.  Responses  to  other  questionnaire  items  suggest 
that  knowledge/skill  deficiencies  in  using  digital  equipment  was  not  cited  by  the  SI  and  S4 
because  (a)  they  were  reasonably  skilled  in  using  the  CSSCS,  and  (b)  a  great  deal  of  the  logistics 
data  had  already  been  entered  into  the  CSSCS  and  remained  constant  throughout  a  mission. 

Second,  six  of  the  nine  battle  staff  members  who  were  actively  involved  in  mission 
planning  indicated  that  planning  difficulties  stemmed  from  the  duplication  of  effort  required  to 
produce  both  digital  products  and  conventional  (analog)  products.  This  reason  was  cited  by  the 
XO,  the  S2,  the  AS2,  the  S3,  one  AS3,  and  the  S4.  The  battle  staff  members  who  did  not  cite 
duplication  of  effort  as  a  reason  for  planning  difficulties  (CO,  SI,  and  FSO)  were  not  actively 
involved  in  the  use  of  digital  equipment  to  produce  planning  products. 

The  other  reasons  cited  for  planning  difficulty  tend  to  be  highly  related  to  the  battle  staff 
members’  responsibilities  or  to  the  digital  equipment  they  were  required  to  use.  Listed  below  are 
the  remaining  reasons  cited  for  planning  difficulty  and  the  battle  staff  member(s)  who  cited  the 
reason: 

•  problems  entering  graphics  and  synchronization  matrices  into  the  MCS  (both 
AS  3  s), 

•  the  icons  on  ASAS  are  too  large  (AS2), 

•  difficulty  communicating  with  brigade  (SI,  FSO), 

•  cannot  track  injured  personnel  adequately  on  CSSCS  (S 1), 

•  more  time  is  spent  getting  better  products  when  digital  systems  are  available 
(CO), 

•  inadequate  time  to  prepare  OPORD  (XO), 

•  the  map  scales  on  the  MCS  and  ASAS  are  far  too  small  for  effective  planning 
(S2,  S3), 

•  lack  of  a  weapons  system  database  (S2), 

•  greater  volume  of  information  with  digital  systems  (SI), 

•  the  MCS  and  ASAS  do  not  provide  reliable  line-of-sight  calculations  (CO, 

S2), 

•  lack  of  SOPs,  enlisted  staff  operations,  and  liaison  with  higher  headquarters 
(XO),  and 

•  it  was  necessary  to  compare  manually  the  information  on  SALUTE  format 
(size,  activity,  location,  unit,  time  and  equipment)  reports  and  the  information 
on  ASAS  reports  (S2). 

Battle  staff  members  cited  a  relatively  small  number  of  reasons  why  mission  planning  in 
the  ATCCS  was  easier  than  planning  a  comparable  operational  mission.  Only  two  reasons  were 
cited  by  more  than  one  battle  staff  member.  One  reason,  cited  by  the  CO,  the  SI,  and  one  AS3, 
was  that  “simulator  rehearsal  is  far  better  than  map  rehearsal.”  The  second  reason,  cited  by  the 
CO,  the  XO,  the  S3,  and  one  AS3,  was  that  the  digital  equipment  (ASAS  and  MCS)  provided  an 
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improved  terrain  analysis  capability.  The  remaining  reasons  are  listed  below  along  with  the 
battle  staff  member  who  cited  the  reason: 

the  environment  is  less  taxing  than  a  field  environment  (XO), 
the  schedule  is  less  taxing  than  a  24-hour  operation  (XO), 
there  are  no  difficult  security  problems  to  deal  with  (XO), 
graphics  are  easy  to  produce  using  the  digital  equipment  (AS3), 
digital  equipment  makes  it  easier  to  convey  coordinates  for  a  large  number  of 
targets  (AS2), 

most  of  the  logistics  data  were  already  entered  into  the  CSSCS  (S4), 
very  easy  to  send  intelligence  messages  to  brigade  with  e-mail  system  (AS 2), 
and 

it  is  easier  to  do  time  calculations  with  the  digital  equipment  (SD3). 

Mission  realism.  Battle  staff  members  rated  the  realism  of  each  mission  using  a  5-point 
rating  scale  that  varied  from  highly  unrealistic  (rating  of  1)  through  moderately  unrealistic 
(rating  of  3)  to  highly  realistic  (rating  of  5).  No  battle  staff  member  rated  any  mission  as  being 
either  highly  unrealistic  (rating  of  1)  or  highly  realistic  (rating  of  5).  Nineteen  ratings  (59.4%) 
indicate  that  the  mission  being  rated  was  judged  to  be  unrealistic  (rating  of  2  or  3).  The 
remaining  13  ratings  (40.6%)  indicate  that  the  mission  being  rated  was  judged  to  be  moderately 
realistic  (rating  of  4).  The  frequency  of  unrealistic  ratings  do  not  differ  significantly  from  the 
realistic  ratings,  %2  (2,  22)  =  1.3,  p  >  .20. 

When  the  ratings  are  pooled  across  battle  staff  members,  the  mean  realism  rating  is 
nearly  identical  for  the  three  missions,  ranging  only  from  3.2  to  3.4.  When  ratings  are  pooled 
across  missions,  the  mean  ratings  of  the  battle  staff  members  range  from  2.3  to  4.0.  The  mean 
realism  rating  is  lowest  for  the  S2  (mean  =  2.3)  and  next  lowest  for  the  SI  and  FSO  (mean  = 

2.7).  The  mean  realism  rating  is  highest  for  the  S4  (mean  =  4.0)  and  nearly  as  high  for  the  CO, 
AS3,  AS2,  and  S3  (mean  =  3.7).  The  remaining  mean  realism  ratings  are  3  (AS3),  3.3  (XO),  and 
3.5  (AS3). 

In  addition  to  the  realism  ratings,  Item  4  asked  battle  staff  members  to  list  ways  the 
mission  was  at  least  moderately  unrealistic.  The  reasons  cited  for  lack  of  realism  are  quite 
similar  to  the  reasons  cited  for  mission  planning  difficulty.  The  lack  of  adequate  staff  training 
was  cited  by  the  XO,  the  S2,  and  the  S3  as  a  reason  for  lack  of  mission  realism.  No  other  reason 
for  lack  of  realism  was  cited  by  more  than  one  battle  staff  member.  Listed  below  are  the  other 
reasons  (other  than  lack  of  adequate  training)  that  were  cited  by  each  battle  staff  member  who 
responded  to  this  item: 

•  CO:  the  communications  were  unrealistically  reliable; 

•  XO:  lack  of  SOPs,  lack  of  enlisted  staff  operations,  not  a  field  environment,  no  24- 
hour  operations,  no  security  problems,  and  no  environmental  problems; 

•  S 1 :  no  direct  feed  from  Forward  Area  Refueling  Point  (FARP)  to  S4,  the  resupply  of 
Class  III  and  Class  IV  was  unrealistically  easy,  and  no  communication  with 
subordinate  CSS  personnel; 
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•  S2:  launched  a  deep  attack  with  too  little  intelligence,  launched  mission  despite 
nearby  threats,  limited  planning  time,  and  missions  too  complex  for  skill  level; 

•  AS2:  a  great  deal  of  duplication  of  effort  in  developing  both  digital  and  analog 
products,  digital  equipment  capabilities  do  not  align  with  MDMP  products,  the  enemy 
artillery  was  largely  unguarded,  and  OPFOR  was  not  as  rational  as  an  actual 
opponent; 

•  S3 :  inadequate  criteria  and  control  measures; 

•  AS3:  no  capability  to  communicate  with  CAS  aircraft; 

•  S4:  unreliable  aircraft  simulation  no  Applique,  and  no  tasks  for  S4  after  input  of  data 
on  fuel  and  ammunition; 

•  FSO:  suitable  maps  were  not  available,  the  rehearsals  were  not  controlled  by 
company  commanders,  and  the  grids  in  the  computer  did  not  correspond  with  the 
grids  on  the  map;  and 

•  ABC:  no  requirement  to  share  resources  or  interact  with  other  units,  and  the  aircraft 
were  exposed  to  an  unrealistically  high  level  of  threat. 

The  data  presented  above  indicate  that,  as  a  group,  the  battle  staff  members  judged  the 
missions  to  be  only  moderately  unrealistic.  However,  the  reasons  for  the  lack  of  realism  do  not 
appear  to  be  related  to  the  realism  of  the  scenarios  for  which  the  missions  were  planned  and 
executed.  Rather,  the  reasons  for  lack  of  realism  centered  on  the  prior  training  of  the  battle  staff 
members  and  the  lack  of  NCOs  to  support  the  battle  staff.  So,  although  battle  staff  members 
indicate  that  the  missions  were  unrealistic  in  some  respects,  neither  the  realism  ratings  nor  the 
reasons  for  lack  of  realism  provide  compelling  evidence  that  the  lack  of  mission  realism  was 
great  enough  to  invalidate  the  findings  of  the  ATCCS  study. 

Adequacy  of  planning  resources.  Items  5,  6,  and  7  asked  battle  staff  members  to  rate 
the  adequacy  of  the  resources  available  to  perform  each  of  seven  MDMP  steps.  The  planning 
resources  included  (a)  the  time  available  to  perform  the  step  (Item  5),  (b)  the  information 
available  to  perform  the  step  (Item  6),  and  (c)  the  accessibility  of  the  information  needed  to 
perform  the  step  (Item  7).  Battle  staff  members  who  did  not  participate  in  an  MDMP  step 
checked  did  not  perform.  Battle  staff  members  who  were  involved  in  performing  an  MDMP  step 
were  asked  to  indicate  whether  the  corresponding  resource  was  adequate  or  inadequate  to 
perform  the  task.  Battle  staff  members  rated  the  adequacy  of  time,  information,  and  information 
accessibility  for  each  of  the  following  MDMP  steps: 

•  Receive  the  Mission, 

•  Analyze  the  Mission, 

•  Develop  the  CO  A, 

•  Analyze  the  COA  (war  gaming), 

•  Compare  the  COA, 

•  Approve  the  COA,  and 

•  Produce  the  orders. 

To  gain  a  clear  understanding  of  battle  staff  members’  views  about  the  adequacy  of 
planning  resources,  it  is  necessary  to  examine  their  judgments  as  a  function  of  mission,  type  of 
resource,  MDMP  step,  and  battle  staff  position.  Throughout  this  subsection,  the  percentage 
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values  refer  to  the  percentage  of  inadequate  ratings  (i.e.,  the  number  of  inadequate  ratings 
divided  by  the  total  number  of  ratings).  Not  every  battle  staff  member  was  involved  in  every 
MDMP  step.  Therefore  the  frequencies  vary  from  one  battle  staff  member  to  another.  Response 
frequencies  for  the  different  battle  staff  members  were  normalized  to  the  same  base  (total 
responses)  prior  to  computing  the  x2  values  that  are  reported  below. 

Figure  1  shows  the  percentage  of  inadequate  ratings  for  time,  information,  and 
information  accessibility  as  a  function  of  the  mission  number.  The  percentage  of  inadequate 
responses  is  not  the  same  for  all  missions  and  planning  resource  types,  %2  (4,  63)  =  36.6,  p< 
.001.  Moreover,  when  responses  are  pooled  across  missions  and  planning  resource  type,  the 
percentage  of  inadequate  responses  differ  as  a  function  of  both  mission,  %2  (2,  63)  =  9.9,  p<  .01, 
and  type  of  planning  resource,  %2  (2,  63)  =  19.7,  p<  .001. 


-Time 

-Information 
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Figure  1.  Percentage  of  inadequate  ratings  as  a  function  of  mission  number  and  type  of  planning 
resource. 

The  percentage  of  inadequate  time  responses  do  not  vary  significantly  as  a  function  of 
mission,  %2  (2,  63)  =  0.5,  p>  .70.  The  percentage  of  inadequate  information  ratings  declined 
significantly  from  the  first  to  the  third  mission,  %2  (2,  63)  =  10.8,  p<  .01.  The  decline  in  the 
percentage  of  inadequate  accessibility  ratings  is  significant  at  the  .  10  level  but  not  at  the  .05 
level,  x2  (2,  63)  =  4.8,/?<  .10. 

The  battle  staff  members’  acquisition  of  knowledge  and  skill  over  the  three  missions  is 
the  most  probable  reason  for  the  decline  in  inadequate  information  and  inadequate  accessibility 
responses  from  the  first  to  the  third  mission.  Apparently,  the  experience  gained  over  the  three 
missions  was  not  enough  to  offset  the  effects  of  inadequate  time  for  performing  some  MDMP 
steps. 
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Because  the  mission  planning  tasks  and  responsibilities  differ  substantially  from  one 
battle  staff  member  to  another  (HQDA,  1997),  it  is  reasonable  to  expect  that  the  adequacy  of 
planning  resources  would  be  judged  differently  by  different  battle  staff  members.  To  examine 
this  relationship,  the  percentages  of  inadequate  ratings  were  calculated  by  battle  staff  position 
and  resource  type.  The  resulting  percentage  values  are  shown  in  Figure  2. 


Inadequacy  Ratings  (Percentage) 


Figure  2.  Percentage  of  inadequate  ratings  as  a  function  of  type  of  planning  resource  and  staff 
position.  Ratings  are  pooled  across  the  seven  MDMP  steps. 

It  should  be  noted  that  no  ratings  are  shown  in  Figure  2  for  the  BC,  the  ABC,  or  the  SI. 
As  mentioned  earlier,  neither  the  BC  nor  the  ABC  was  required  to  participate  actively  in  mission 
planning.  Although  the  SI  was  considered  a  battle  staff  member  in  mission  planning,  his 
questionnaire  responses  indicate  that  he  performed  only  one  MDMP  step  for  one  mission.  The 
AS2,  the  S3,  and  one  AS3  indicated  that  they  were  involved  in  performing  all  seven  of  the 
MDMP  steps  for  each  of  the  three  missions.  All  other  battle  staff  members  (CO,  XO,  S2,  S4, 
and  FSO)  indicated  that  they  performed  three  or  more  of  the  MDMP  steps  during  every  mission. 

First,  consider  the  judgments  about  the  adequacy  of  planning  time.  The  percentage  of 
inadequate  time  responses  differ  significantly  across  battle  staff  members,  %2  (7,  21)  =  32.9,  p< 
.001.  The  percentage  of  inadequate  time  ratings  is  about  50%  for  the  XO,  the  S3,  and  the  S4;  the 
ratings  for  the  XO,  the  S3,  and  the  S4  do  not  differ  significantly,  %2  (2,  21)  =  0.02,  p>  .80.  The 
percentages  of  inadequate  time  ratings  are  23.3%  for  the  S2  and  12.5%  or  lower  for  the 


36 


remaining  battle  staff  members.  The  percentage  of  inadequate  time  ratings  is  significantly 
higher  for  the  XO,  the  S3,  and  the  S4  (as  a  group)  than  for  the  remaining  battle  staff  members  (as 
a  group),  x2  (1,  21)  =  6.5,  p<  .02. 

Because  the  XO,  the  S3,  and  the  S4  must  contribute  to  the  performance  of  many  MDMP 
steps,  it  is  not  surprising  that  about  one-half  of  their  time  judgments  were  inadequate.  Based  on 
the  number  of  MDMP  steps  to  which  the  S2  must  contribute,  it  would  be  expected  that  the 
percentage  of  inadequate  time  judgments  would  have  been  higher.  A  possible  reason  for  the 
S2’s  responses  is  that  the  volume  of  intelligence  information  that  was  available  for  mission 
planning  was  unrealistically  low. 

An  analysis  was  performed  to  determine  whether  the  percentage  of  inadequate  time 
ratings  was  the  same  for  the  different  MDMP  steps.  Considering  the  XO,  the  S2,  the  S3,  and  S4 
as  a  group,  it  was  found  that  67.9%  of  the  inadequate  time  ratings  are  for  three  steps:  Develop 
the  COA,  Analyze  the  COA,  and  Compare  the  CO  A.  More  will  be  said  later  about  the  adequacy 
of  planning  resources  for  performing  the  different  MDMP  steps. 

Next,  consider  battle  staff  members’  judgments  about  the  adequacy  of  the  mission 
planning  information.  The  percentage  of  inadequate  information  responses  differ  significantly 
across  battle  staff  members,  %2  (7,  21)  =  23.0,  p<  .01.  The  percentage  of  inadequate  information 
responses  is  large  (47.6%  to  52.4%)  and  not  significantly  different  for  the  XO,  the  S2,  the  S3, 
and  the  S4,  jf  (7,  21)  =  0.1,  p>  .80.  The  percentages  of  inadequate  information  responses  are 
28.6%  forthe  AS2,  21.4%  for  the  XO,  and  0.0%  for  both  the  AS3  and  the  FSO.  Given  their 
roles  in  many  of  the  MDMP  steps,  it  is  not  surprising  to  find  that  the  percentages  of  inadequate 
ratings  are  high  for  the  XO,  the  S2,  the  S3,  and  the  S4. 

Finally,  consider  the  judgments  of  information  accessibility.12  As  is  true  for  time 
adequacy  and  information  adequacy,  the  percentage  of  inadequate  accessibility  responses  differ 
significantly  across  battle  staff  members,  %2  (7,  21)  =  18.7  ,p<  .01.  Figure  2  shows  that  the 
percentages  of  inadequate  accessibility  responses  range  from  a  high  of  90.5%  (XO)  to  a  low  of 
12.2%  (AS3).  Figure  2  shows  that  the  percentages  of  inadequate  accessibility  are  quite  large 
(42.8%  to  90.5%)  for  all  battle  staff  members  except  the  AS3  (12.2%)  and  the  CO  (28.6%).  If 
the  AS3  and  CO  are  excluded,  the  percentages  of  inadequate  accessibility  responses  for  the 
remaining  battle  staff  members  do  not  differ  significantly,  x2  (6,  21)  =  11.0,  p>  .05. 

Figure  2  indicates  that  the  XO,  the  S3,  the  FSO,  and  the  AS2  judged  information 
accessibility  to  be  more  of  a  problem  (higher  percentage  of  inadequate  responses)  than  either 
time  availability  or  information  availability.  However,  the  differences  are  significant  only  for 
the  FSO,  x2  (2,  21)  =  12.8,  p<  .01,  and  the  AS2  %2  (2,  21)  =  7.4,  p<  .05. 

Figure  3  shows  the  percentage  of  inadequate  ratings  as  a  function  of  MDMP  step  and 
type  of  planning  resource.  The  percentage  values  were  calculated  by  pooling  the  ratings  of  all 
battle  staffmembers  who  participated  in  mission  planning.  As  is  apparent  in  Figure  3,  the 

1 :  In  this  study,  the  only  planning  information  that  was  not  easily  accessible  was  that  which  could  be  acquired  only 
through  the  digital  systems.  So,  judgements  of  inadequate  information  accessibility  are  assumed  to  be  the  result  of 
problems  encountered  in  acquiring  information  through  the  digital  systems. 
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percentage  of  inadequate  responses  is  not  the  same  for  all  MDMP  steps  and  all  types  of  planning 
resources,  %2  (7,  504)  =  44.2,  p<  .001.  A  %2  analysis  shows  that  the  number  of  inadequate 
ratings  differ  across  MDMP  steps  for  time,  %2  (7,  168)  =  34.9,  p<  .001,  information,  %2  (7,  168) 
=  41.  \,p<  .001,  and  accessibility,  X2  (7,  168)  =  58.0, p<  .001. 

The  percentage  of  inadequate  ratings  vary  as  a  function  of  resource  type  for  three  MDMP 
steps:  Receive  the  Mission,  %2  (2,  72)  =  5.6,  p<  .02,  Analyze  the  Mission,  %2  (2,  72)  =  8.7,  p< 
.01,  and  Develop  the  COA,  %2  (2,  72)  =  4.2,  p<  .05.  For  the  remaining  MDMP  steps,  the 
percentage  of  inadequate  responses  do  not  differ  significantly  (p>  .05)  across  the  three  resource 
types. 

For  the  most  part,  the  data  shown  in  Figure  3  are  self-explanatory.  However,  three  findings 
deserve  a  brief  comment.  First,  it  is  worth  noting  that  for  every  MDMP  step  the  percentage  of 
inadequate  accessibility  responses  is  larger  than  the  percentage  of  inadequate  time  responses  and 
the  percentage  of  inadequate  information  responses. 
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Figure  3.  Percentage  of  inadequate  ratings  as  a  function  of  type  of  planning  resource  and 
MDMP  step.  Ratings  are  pooled  across  all  battle  staff  members  who  were  involved  in  mission 
planning. 

Although  not  all  of  the  differences  are  statistically  significant,  this  finding  is  consistent  with  the 
observation  made  earlier  that  inadequate  accessibility  of  information  was  a  somewhat  greater 
problem  than  either  inadequate  time  or  inadequate  information. 

Second,  it  is  worth  noting  that  the  percentages  of  inadequate  time  responses  are  relatively 
high  for  the  MDMP  steps  Develop  the  COA  (23.8%),  Analyze  the  COA  (42.9%),  Compare  the 
COA  (36.8%),  and  Produce  the  Orders  (29.2%).  Because  of  the  number  of  substeps  that  must  be 
completed,  it  is  not  surprising  the  percentages  of  inadequate  time  responses  are  highest  for  the 
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tasks  Develop  the  COA,  Analyze  the  CO  A,  and  Compare  the  CO  A.  However,  it  was  surprising 
to  find  a  relatively  large  percentage  of  inadequate  time  responses  for  the  MDMP  step  Produce 
the  Orders.  Because  producing  orders  is  not  nearly  as  complex  and  labor  intensive  as  some  of 
the  other  MDMP  steps,  it  is  possible  that  the  battle  staff  members  used  most  of  the  available  time 
performing  other  MDMP  steps  and  left  an  inadequate  amount  of  time  for  the  final  step:  Produce 
the  Orders.  In  short,  it  seems  likely  that  a  time  management  problem  was  the  reason  for  the 
inadequate  time  responses  for  the  MDMP  step  Produce  the  Orders. 

Third,  as  would  be  expected,  the  percentages  of  inadequate  information  ratings  are 
relatively  high  for  all  the  MDMP  steps  for  which  the  compilation  of  information  is  a  critical 
requirement:  Analyze  the  Mission  (33.3),  Develop  the  COA  (36.4%),  Analyze  the  COA 
(52.2%),  and  Compare  the  COA  (33.3%).  The  reasons  for  the  small  but  nontrivial  percentages 
of  inadequate  information  responses  for  the  other  three  MDMP  steps  are  not  known  for  certain. 
The  inadequate  information  responses  for  the  MDMP  step  Receive  the  Order  (16.7%)  may  be 
due  to  the  problems  encountered  in  receiving  the  OPORD  on  the  MCS  (on  Day  1).  The 
inadequate  information  responses  for  the  MDMP  steps  Approve  the  COA  (12.5%)  and  Produce 
the  Orders  (16.7%)  may  be  due  to  the  problems  encountered  in  using  the  digital  systems  to 
produce  briefing  materials  and  to  produce  written  orders. 

Workload  level.  Ail  item  asking  battle  staff  members  to  rate  their  level  of  workload  was 
included  on  the  questionnaire  with  the  full  realization  that  responses  to  such  items  have  not 
proved  to  be  highly  reliable  or  highly  sensitive  measures  of  mental  workload.  Unfortunately,  the 
methods  that  are  known  to  provide  reliable  methods  of  workload  were  judged  too  time 
consuming  and  too  obtrusive  for  this  study.  For  a  review  of  subjective  workload  assessment 
techniques,  see  (Christ  et  al.,  1993;  Lysaght  et  al.,  1989;  Schmidt  and  Nicewonger,  1988; 
Wierwille,  Rahimi,  and  Casalli,  1985).  For  this  reason,  it  is  important  to  exercise  caution  in 
interpreting  battle  staff  members’  workload  ratings.  In  particular,  the  battle  staff  members’ 
workload  ratings  reported  below  should  not  be  used  to  draw  inferences  about  the  presence  or 
absence  of  excessive  workload  (i.e.,  mental  overloads)  for  individual  battle  staff  members. 
However,  it  seems  safe  to  use  the  ratings  to  draw  inferences  about  relative  workload  from  one 
mission  to  another  and  from  one  MDMP  step  to  another. 

Item  8  asked  battle  staff  members  to  rate  the  highest  level  of  workload  they  experienced 
when  performing  each  of  seven  MDMP  steps.  Battle  staff  members  used  a  rating  scale  with  four 
levels:  Too  Low,  Moderate,  High,  and  Very  High.  Battle  staff  members  were  asked  to  place  a 
check  in  a  separate  box  if  they  were  not  involved  in  performing  the  corresponding  MDMP  step. 
Because  some  battle  staff  members  were  not  involved  in  mission  planning,  a  total  of  198 
workload  ratings  resulted  from  the  three  missions,  the  seven  MDMP  steps.  The  distribution  of 
ratings  for  the  total  sample  is  shown  below. 

•  Too  Low  =  3.5% 

•  Moderate  =  51.5% 

•  High  =  31.9% 

•  Very  High  =  13.1% 
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The  results  of  a  %2  test  indicate  that  the  four  rating  values  were  not  selected  with  equal 
frequency,  x2  (3,  198)  =  96.9 ,p<  .001.  An  analysis  of  the  ratings  across  missions  showed  no 
change  in  the  relative  frequency  across  missions  for  the  moderate  rating,  %2  (2,  66)  =  2.3,  p>  .05, 
and  the  high  rating,  x2  (2,  66)  =  1.0,  p>  .05.  The  frequency  of  too  low  ratings  declined 
significantly  from  Mission  1  (10.6%)  to  Mission  2  (0.0%),  x2  (2,  66)  =  14.0,  p<  .001.  The 
frequency  of  very  high  ratings  also  declined  from  Mission  1  (21.2%)  to  Mission  3  (8.7%),  but  the 
decline  failed  to  reach  the  .05  level  of  significance,  y2  (2,  66)  =  5.1  ,p<  .10.  The  decline  in  both 
too  low  and  very  high  ratings  suggests  that  the  experience  gained  in  Mission  1  enabled  battle 
staff  members  to  become  more  proficient  at  distributing  the  work  such  that  no  battle  staff 
member  had  too  little  or  too  much  to  do. 

Too  low  workload  ratings  are  important  because  they  suggest  that  a  battle  staff  member 
may  be  underutilized.  Only  8  of  the  196  workload  ratings  are  too  low  and  all  too  low  workload 
ratings  were  given  by  three  battle  staff  members:  the  XO,  the  BC,  and  the  SI.  The  data 
presented  below  show  that  the  XO  clearly  was  not  underutilized.  The  BC  was  not  actively 
involved  in  the  planning  process,  so  his  too  low  ratings  indicate  only  that  his  workload  was  low 
for  the  few  MDMP  steps  in  which  he  participated.  The  SI  was  an  active  member  of  the  planning 
team,  so  his  too  low  workload  ratings  suggest  that  he  was  underutilized  during  mission  planning. 
This  conclusion  is  supported  by  the  fact  that  the  SI  gave  no  workload  rating  higher  than 
moderate.  The  reasons  for  the  Si’s  under-utilization  during  mission  planning  are  not  known. 

The  distribution  of  high  and  very  high  workload  ratings  across  battle  staff  members 
provides  an  indication  of  unequal  and,  perhaps,  inefficient  distribution  of  work  among  battle 
staff  members.  Reference  to  FM  101-5  (HQDA,  1997)  shows  that  battle  staff  members  vary 
widely  in  the  number  of  MDMP  steps  and  substeps  to  which  a  battle  staff  member  is  expected  to 
contribute.  In  short,  an  unequal  distribution  of  mission  planning  work  may  be  dictated  by 
doctrine.  This  conclusion  is  supported  by  Figure  4,  which  shows  the  percentage  of  high  ratings, 
the  percentage  of  very  high  ratings,  and  the  combined  percentage  of  high  and  very  high  ratings 
for  the  battle  staff  members  who  were  actively  involved  in  mission  planning. 

Analyses  were  conducted  to  test  the  null  hypothesis  that  the  percentages  of  high 
workload  ratings  and  the  percentages  of  very  high  workload  ratings  are  the  same  for  all  battle 
staff  members.  Statistically  significant  differences  among  battle  staff  members  were  found  for 
the  percentages  of  high  ratings,  %2  (6,  141)  =  38.5,  p<  .001,  the  percentages  of  very  high  ratings, 
X2  (6,  141)  =  3 1.0,  p<  .001,  and  the  combined  percentages  (high  plus  very  high  ratings,  %2  [6, 
141]  =  69.4,  p<  .001). 
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Figure  4.  Percentage  of  high,  very  high,  and  combined  (high  plus  very  high)  workload  ratings 
for  seven  battle  staff  members. 

Figure  4  suggests  that  the  battle  staff  members  can  be  separated  into  two  groups  with 
respect  to  the  combined  percentages  of  high  and  very  high  ratings.  This  grouping  is  supported 
by  the  results  of  a  y2  analysis.  The  combined  percentages  for  the  S2,  the  S3,  and  the  S4  do  not 
differ  significantly,  y2  (2,  58)  =  0.4,  p>  0.90.  Similarly,  the  combined  percentages  for  the  XO, 
the  AS2,  the  AS3,  and  the  FSO  do  not  differ  significantly,  y2  (4,  85)  =  0.3,  p>  0.90.  However, 
the  mean  combined  percentage  for  the  S2,  the  S3,  and  the  S4  is  significantly  larger  than  the  mean 
combined  percentage  for  the  XO,  the  AS2,  the  AS3,  and  the  FSO,  y2  (2,  141)  =  4.0,  p<  .05. 

The  data  shown  in  Figure  4  suggest  that  work  was  not  distributed  uniformly  among  the 
battle  staff  members.  Because  of  the  large  number  of  MDMP  substeps  in  which  they  are 
involved,  it  is  not  surprising  that  the  combined  percentages  of  high  and  very  high  workload 
ratings  are  relatively  large  for  both  the  S2  and  S3.  It  is  interesting  to  note  that  the  combined 
percentages  for  the  S2  and  S3  are  substantially  larger  than  the  combined  percentages  for  then- 
assistants,  the  AS2  and  AS3.  Apparently,  the  S2  and  the  S3  were  unable  or  unwilling  to  off-load 
a  sufficient  amount  of  their  planning  tasks  to  their  assistants.  It  was  surprising  to  find  that  every 
one  of  the  S4’s  workload  ratings  are  high  or  very  high,  the  reasons  for  high  workload  level 
reported  by  the  S4  are  not  known. 

An  analysis  was  conducted  to  test  the  null  hypothesis  that  the  percentage  of  responses 
selecting  each  workload  rating  was  the  same  for  all  MDMP  steps.  It  was  found  that  that  the 
percentage  of  low  workload  ratings  do  not  vary  as  a  function  of  MDMP  step,  y2  (6,  8)  =  9. 1,  p> 
.20.  However  differences  vary  as  a  function  of  MDMP  step  for  moderate  ratings,  y2  (6,  99)  = 
71.5 ,p<  .001,  high  ratings,  y2  (6,  99)  =  65.5 ,p<  .001,  and  very  high  ratings  y2  (6,  99)  =  21.4, p< 
.001.  Figure  5  shows  the  percentage  of  high  ratings,  very  high  ratings,  and  combined  high  and 
very  high  ratings  for  each  MDMP  step. 
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Of  the  seven  MDMP  steps,  the  reports  of  heavy  workload  were  most  frequent  for  the 
MDMP  step  Analyze  the  COA.  Combined,  the  high  and  very  high  ratings  account  for  86.3%  of 
all  workload  ratings  given  for  this  step.  The  combined  percentages  of  high  and  very  high 
workload  ratings  are  also  quite  large  for  the  steps  Analyze  the  Mission  (40.0%),  Develop  the 
COA  (56.5%),  Compare  the  COA  (50.0%),  and  Produce  the  Orders  (50.0%).  Although  the 
combined  percentages  for  these  five  MDMP  steps  differ  by  45.4%,  the  differences  are  not  large 
enough  to  reach  statistical  significance  at  the  .05  level,  x2  (4,  137)  =  4.5,  p>  .30. 

By  comparison,  the  combined  percentages  of  high  and  very  high  workload  ratings  are 
relatively  small  for  the  MDMP  steps  Receive  the  Mission  (20.0%)  and  Approve  the  COA 
(20.0%).  The  mean  combined  percentage  for  the  MDMP  steps  Receive  the  Mission  and 
Approve  the  COA  (20.0%)  differ  significantly  from  the  mean  combined  percentage  for  the  other 
five  MDMP  steps  (56.0%),  x2  (2,  196)  =  5.4,  p<  .05. 
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Figure  5.  Percentages  of  high,  very  high,  and  combined  {high  plus  very  high )  workload  ratings 
for  seven  MDMP  steps. 
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Figure  6.  Relationship  between  the  percentage  of  inadequate  time  ratings  and  heavy  ( high  plus 
very  high)  workload  ratings. 

High  workload  can  result  from  task  complexity,  time  constraints,  or  a  combination  of  the 
two.  If  time  constraint  was  the  main  contributor  to  battle  staff  members’  workload,  the 
percentage  of  high  workload  ratings  should  be  closely  related  to  the  percentage  of  inadequate 
time  ratings.  Figure  6  shows  the  percentage  of  inadequate  time  ratings  and  the  percentage  of 
heavy  workload  ratings  {high  plus  very  high  workload)  for  each  of  the  MDMP  steps.  A 
Spearman  rho  coefficient  (Guilford  and  Fructer,  1978)  is  significant  at  the  .05  level,  r  (6  df)  = 
0.9,  p<  .05,  indicating  that  the  inadequate  time  ratings  are  highly  related  to  the  heavy  workload 
ratings. 


Although  the  two  ratings  are  closely  related,  the  percentage  of  heavy  {high  plus  very 
high)  workload  ratings  is  considerably  larger  than  the  percentage  of  inadequate  time  ratings  for 
every  MDMP  step.  The  differences  range  from  a  low  of  13.2%  (Compare  the  COA)  to  a  high  of 
43.5%  (Analyze  the  COA).  These  differences  between  the  two  types  of  ratings  may  reflect  the 
extent  to  which  factors  other  than  time  constraints  contributed  to  battle  staff  members’  workload. 
This  interpretation  is  supported  by  the  finding  that  the  largest  differences  between  inadequate 
time  ratings  and  heavy  workload  are  found  for  the  three  MDMP  steps  that  require  the  most 
experience  and  analytic  skills:  Analyze  the  COA  (difference  —  43.5%),  Analyze  the  Mission 
(difference  =  35.5%),  and  Develop  the  COA  (difference  =  32.7%). 

Planning  errors  and  oversights.  For  each  of  seven  MDMP  steps,  battle  staff  members 
were  asked  if  errors  or  oversights  were  made  in  performing  the  step.  Battle  staff  members  who 
responded  affirmatively  were  asked  to  identify  each  error  or  oversight  that  was  made  and  to 
identify  the  step  in  which  it  was  made.  The  percentages  of  affirmative  responses  for  each  of  the 
three  missions  are  shown  below.  Although  the  percentage  of  affirmative  responses  declined 
from  the  first  to  the  third  mission,  the  decline  was  not  great  enough  to  reach  statistical 
significance  at  the  .05  level,  %2  (2,  60)  =  3.3),  p>  .05. 
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•  Mission  1  =  43.3% 

•  Mission  2  =  29.1% 

•  Mission  3  =  25.5% 

The  percentages  of  affirmative  responses  vary  substantially  among  the  battle  staff 
members  who  participated  actively  in  mission  planning,  %2  (7, 21)  =  61.7 ,p<  .001.  Figure  7 
shows  the  percentage  of  affirmative  responses  for  each  of  the  seven  battle  staff  members  who 
was  an  active  battle  staff  member  in  mission  planning.  The  percentages 
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Figure  7.  Self-reported  planning  errors/oversights  for  the  battle  staff  members  who  were  actively 
involved  in  mission  planning. 

shown  in  Figure  7  are  based  on  the  total  number  of  responses  made  for  the  three  missions  and  the 
seven  MDMP  steps  (n  =  21). 

The  frequencies  of  affirmative  responses  are  relatively  high,  but  not  statistically  different, 
X2  (2,  21)  =  0.5,  p>  .8,  for  the  three  battle  staff  members  who  participated  in  the  largest  number 
of  MDMP  steps  and  substeps:  the  XO  (66.7%),  the  S2  (66.7%),  and  the  S3  (52.4%).  As  a 
group,  the  XO,  the  S2,  and  S3  reported  a  larger  number  of  planning  errors  and  oversights  than 
the  remaining  battle  staff  members  (as  a  group),  %2  (2,  21)  =  8.1,/?<  .01.  The  percentage  of 
affirmative  responses  is  relatively  low  for  the  CO  (27.3%),  the  AS2  (0.0%),  the  AS3  (4.8%),  the 
S4  (14.3%),  and  the  FSO  (0.0%);  the  percentages  do  not  differ  significantly  among  these  battle 
staff  members,  x2  (3,  21)  =  6.1,  p>  .10. 

The  larger  number  of  planning  errors  and  oversights  reported  by  the  XO,  S2,  and  S3  does 
not  reflect  a  lessor  level  of  experience  or  skill  than  the  remaining  battle  staff  members.  The  XO, 
S2,  and  S3  were  at  least  as  experienced  as  all  other  battle  staff  members  except  the  CO.  Rather, 
the  large  number  of  reported  planning  errors/oversights  is  probably  due  to  the  fact  that  the  XO, 
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S2,  and  S3  were  in  a  better  position  to  recognize  planning  errors  and  oversights  because  of  their 
involvement  in  most  of  the  MDMP  steps  and  substeps. 

The  percentages  of  affirmative  responses  are  shown  by  MDMP  step  in  Figure  8.  The 
percentages  in  Figure  8  are  based  on  the  responses  made  for  three  missions  by  the  eight  battle 
staff  members  who  participated  actively  in  mission  planning:  CO,  XO,  S2,  AS2,  S3,  AS3,  S4, 
and  FSO  («  =  21).  Although  the  range  of  percentage  values  range  from  a  low  of  21.1%  (Receive 
the  Mission  and  Approve  the  COA)  to  a  high  of  66.7%  (Develop  the  CO  A),  the  percentages  do 
not  differ  significantly,  x2  (6,  21)  =  6.9,  p>  .5. 
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Figure  8.  Self-reported  planning  errors/oversights  for  each  of  the  seven  MDMP  steps. 

Although  the  number  of  reported  planning  errors  and  oversights  does  not  differ 
significantly  from  one  MDMP  step  to  another,  the  absolute  sizes  of  the  percentage  values  for 
some  MDMP  steps  are  worthy  of  comment.  First,  the  number  of  planning  errors  and  oversights 
reported  for  the  MDMP  steps  Receive  the  Mission  and  Produce  the  Orders  are  surprisingly  large. 
Although  the  reasons  for  the  large  percentage  values  are  not  known,  it  is  probable  that  the  errors 
made  in  Receiving  the  Mission  and  Producing  the  Orders  were  technical  rather  than  conceptual. 
Comments  by  the  battle  staff  members  indicate  that  they  encountered  difficulties  in  using  the 
digital  systems  to  receive  the  OPORD  and  to  prepare  the  written  orders  for  a  mission.  Most  of 
the  remaining  MDMP  steps  are  dependent  on  both  conceptual  knowledge  and  skill  in  using  the 
digital  systems.  Hence,  it  seem  likely  that  both  conceptual  errors  and  technical  errors 
contributed  to  the  errors  and  oversights  reported  for  the  MDMP  steps  Analyze  the  Mission, 
Develop  the  COA,  Analyze  the  COA,  and  Compare  the  COA.  Data  presented  in  the  next 
subsection  support  the  above  speculation  that  both  conceptual  and  technical  errors  and  oversights 
contributed  to  the  battle  staff  members'  reports. 

Although  all  battle  staff  members  were  requested  to  describe  the  planning  errors  and 
oversights  they  observed,  only  a  few  battle  staff  members  provide  meaningful  descriptions. 
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Listed  below  are  the  errors/oversights  described  on  the  questionnaire.  The  battle  staff  member 
who  described  the  error  is  shown  in  parentheses  at  the  end  of  each  description. 

•  The  doctrinal  MDMP  was  modified  to  accommodate  the  limited  capabilities  of  the 
digital  systems.  This  resulted  in  an  incomplete,  spotty  analysis.  (S2,  AS2) 

•  Terrain  analysis  errors  occurred  because  of  the  lack  of  a  large-scale  map.  Errors  also 
were  made  in  performing  line-of-sight  analyses  because  of  the  lack  of  a  large-scale 
map  and  limitations  in  the  digital  systems.  (CO,  S2) 

•  Because  of  digital  system  limitations,  errors  were  made  in  producing  the  modified 
combined  obstacle  overlay  (MCOO).  (AS2) 

•  Many  MDMP  steps  were  not  performed  adequately  or  were  not  performed  at  all  (e.g., 
Develop  the  CO  A,  Analyze  the  CO  A,  and  Compare  the  COA).  (XO,  S2) 

•  The  attack  battle  maneuver  was  not  doctrinal.  (S3) 

•  Did  not  develop  a  CSAR  plan.  (S3,  S4) 

•  Did  not  do  adequate  contingency  planning.  (S3) 

•  Developed  and  used  nonstandard  graphics.  (S3) 

•  Did  not  provide  command  guidance  early  enough  in  the  planning  process.  (CO) 

•  Did  not  recognize  the  need  for  a  jump  FARP.  (AS3) 

Training  deficiencies.  Item  10  asked  battle  staff  members  to  indicate,  for  each  MDMP 
step,  whether  or  not  lack  of  familiarity  with  the  digital  system  made  it  more  difficult  to  perform 
the  step.  For  each  of  six  MDMP  steps,13  Item  14  asked  battle  staff  members  to  indicate  whether 
their  prior  training  was  adequate  for  performing  the  step.  Battle  staff  members’  prior  training 
influenced  their  responses  to  both  items.  Deficiencies  in  digital  system  training  influenced  battle 
staff  members’  responses  to  Item  10,  while  training  deficiencies  in  general  influenced  their 
responses  to  Item  14,  including  deficient  training  on  the  digital  systems.  Because  of  the 
similarity  between  Item  10  and  Item  14,  battle  staff  members’  responses  to  the  two  items  are 
discussed  together.  The  percentage  values  presented  below  refer  to  the  percentage  of  inadequate 
responses:  inadequate  familiarity  with  the  digital  systems  and  inadequate  prior  training  in 
general.14 

Table  4  shows  the  percentage  of  inadequate  familiarity  responses  and  the  percentage  of 
inadequate  prior  training  responses  for  each  of  the  three  missions.  The  percentages  of 
inadequate  familiarity  responses  range  from  53.3%  to  69.1%.  Although  the  percentage  of 
inadequate  familiarity  responses  declined  24.5%  from  Mission  1  (69.1%)  to  Mission  2  (44.6%), 
the  percentages  for  the  three  missions  do  not  differ  significantly,  %2  (2,  68)  =  3.8,  p>  .10.  The 
percentage  of  inadequate  prior  training  responses  is  relatively  uniform  across  missions,  varying 
from  48.2%  to  56.8%.  The  percentages  of  inadequate  prior  training  responses  do  not  differ 
significantly  across  the  three  missions,  %2  (2,  56)  =  0.1,  p>  .90.  The  data  shown  in  Table  4 


13  The  MDMP  step  Produce  the  Orders  was  inadvertently  excluded  from  the  list  of  MDMP  steps  presented  in  Item 
14. 

14  Item  10  asked  battle  staff  members  if  lack  of  familiarity  with  the  digital  systems  made  it  more  difficult  to  perform 
a  task.  A  “yes”  response  was  interpreted  to  mean  that  familiarity  with  the  digital  systems  was  inadequate.  A  “no” 
response  was  interpreted  to  mean  that  familiarity  with  the  digital  systems  was  adequate.  It  is  possible  that  some  of 
the  “no”  responses  indicated  that  the  digital  systems  were  not  used  to  perform  the  planning  task,  so  lack  of 
familiarity  with  them  did  not  influence  task  performance. 
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suggest  that  the  experience  gained  during  the  three  missions  did  little  to  offset  the  deleterious 
effects  of  inadequate  familiarity  with  the  digital  systems  and  inadequate  prior  training  in  general. 

Table  4 

Percentage  of  inadequate  familiarity  responses  and  inadequate  prior  training  responses  for  each 
of  three  missions. 


Mission  Number 

Type  of  Training  Deficiency 

1 

2 

3 

Inadequate  Familiarity  with  Digital  Systems  ( n  =  68) 

69.1% 

44.6% 

53.3% 

Inadequate  Prior  Training,  General  (n  =  56) 

52.1 

48.2 

56.8% 

Note.  The  numbers  (ri)  represent  the  total  number  of  responses  for  a  mission,  pool 

ed  across 

MDMP  steps  and  battle  staff  members. 

The  percentages  of  inadequate  familiarity  responses  and  the  percentages  of  inadequate 
prior  training  responses  are  shown  for  individual  MDMP  steps  in  Figure  9.  The  MDMP  step 
Produce  the  Orders  was  inadvertently  excluded  from  Item  14.  Although  the  percentage  of 
inadequate  familiarity  responses  range  from  a  low  of  28%  (Receive  the  Mission)  to  a  high  of 
75.0%  (Develop  the  CO  A),  the  differences  in  percentages  across  MDMP  steps  are  not 
statistically  significant,  x2  (6,  68)  =  S2,p>  .40.  The  range  of  percentage  values  is  slightly  less 
for  the  inadequate  prior  training  responses  (from  38.5%  to  75.0%)  and,  again,  the  differences  in 
percentages  across  MDMP  steps  are  not  statistically  significant,  %2  (5,  56)  =  4.8, /»  .30. 

Although  neither  inadequate  familiarity  responses  nor  inadequate  prior  training 
responses  differ  significantly  across  MDMP  steps,  the  absolute  sizes  of  several  percentage  values 
are  worthy  of  brief  comment.  A  relatively  large  percentage  of  inadequate  responses  would  be 
expected  for  the  four  MDMP  steps  that  require  the  largest  number  of  substeps  and  most  complex 
analyses  (viz.,  Analyze  the  Mission,  Develop  the  CO  A,  Analyze  the  CO  A,  and  Compare  the 
CO  A).  However,  it  must  be  considered  surprising  that  (a)  the  percentages  of  inadequate 
familiarity  responses  for  these  four  MDMP  steps  are  no  less  than  54.5%,  and  (b)  the  percentages 
of  inadequate  prior  training  responses  are  no  less  than  46.2%.  A  substantially  smaller  percentage 
of  inadequate  responses  would  be  expected  for  the  MDMP  steps  that  require  fewer  substeps  and 
that  require  a  lessor  amount  of  complex  analyses  to  perform.  However,  Figure  9  shows  that  the 
percentages  of  inadequate  familiarity  responses  and  the  percentages  of  inadequate  prior  training 
responses  are  quite  large  for  the  MDMP  steps  Receive  the  Mission  and  Approve  the  CO  A;  the 
percentage  of  inadequate  familiarity  responses  also  is  quite  large  for  the  MDMP  step  Develop 
the  Orders.  Problems  encountered  in  using  the  digital  systems  to  perform  largely  clerical  tasks 
may  account  for  the  large  percentages  of  inadequate  familiarity  responses  for  these  three  MDMP 
steps.  However,  no  reasons  can  be  given  for  the  large  percentages  of  inadequate  prior  training 
responses  given  for  the  MDMP  steps  Receive  the  Mission  and  Approve  the  CO  A. 
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Figure  9.  Percentages  of  inadequate  familiarity  responses  and  inadequate  prior  training 
responses  for  each  MDMP  step. 


Figure  10  shows  the  percentage  of  inadequate  familiarity  responses  and  the  percentage  of 
inadequate  prior  training  responses  for  each  of  the  battle  staff  members  who  were  active  battle 
staff  members  in  mission  planning.  The  FSO  failed  to  respond  to  Item  10  and  Item  14  because 
of  a  misinterpretation,  so  was  not  included  in  Figure  10.  The  percentages  of  inadequate 
familiarity  responses  differ  significantly  across  battle  staff  members,  x2  (6,  21)  =  35.4,  p<  .001. 
Figure  10  shows  that  the  battle  staff  members  can  be  divided  into  two  groups  based  upon  the 
percentages  of  inadequate  familiarity  responses.  The  low  percentage  group  has  only  two 
members,  the  CO  (27.3%)  and  the  AS3  (28.6%).  The  high  percentage  group  includes  all  other 
members  of  the  battle  staff:  the  XO  (66.7%),  the  S2  (65%),  the  AS2  (50%),  the  S3  (76.2%),  and 
the  S4  (92.9%).  The  percentages  of  members  of  the  high  percentage  group  do  not  differ 
significantly,  %2  (4,  21)  =  2.98  ,p>  .30.  However,  the  percentages  of  inadequate  familiarity 
responses  are  significantly  larger  for  the  high  group  than  the  low  group,  x2  (1,  21)  =  3.8,  p<  .05. 
With  one  exception — the  AS3 — the  percentage  of  inadequate  familiarity  responses  appear  to  be 
related  to  the  battle  staff  member’s  use  of  the  digital  system.  The  low  percentage  of  inadequate 
familiarity  responses  by  the  AS3  may  be  due  to  his  higher  level  of  familiarity  with  the  digital 
system  or  to  his  use  of  the  digital  systems  only  for  tasks  he  knew  how  to  perform. 

The  percentages  of  inadequate  prior  training  responses  differ  significantly  across  battle 
staff  members,  %2  (6,  21)  =  25.4  ,p<  .001.  However,  if  the  AS3’s  percentage  is  excluded,  the 
percentages  of  inadequate  prior  training  responses  for  the  remaining  members  of  the  battle  staff 
do  not  differ  significantly,  x2  (5,  21)  =  2.83,  p>  .70.  Excluding  the  AS3,  the  percentages  of 
inadequate  prior  training  responses  range  from  a  low  of  66.7%  to  a  high  of  100%.  The  data 
shown  in  Figure  10  suggest  that  inadequate  prior  training  adversely  influenced  the  performance 
of  every  battle  staff  member  except  the  AS3.  The  prior  training  of  the  AS3  was  similar  to  that  of 
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other  staff  members.  Therefore  the  low  percentage  of  inadequate  prior  training  responses 
almost  certainly  is  due  to  the  fact  that  the  AS3  was  assigned  less  demanding  tasks  than  the  other 
members  of  the  battle  staff 
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Figure  10.  Percentage  of  inadequate  familiarity  responses  and  inadequate  prior  training 
responses  for  each  battle  staff  member  who  was  an  active  battle  staff  member  in  mission 
planning. 

Both  Item  10  and  Item  14  asked  battle  staff  members  to  describe  specific  knowledge  and 
skill  deficiencies  that  degraded  their  mission  planning  performance.  Every  battle  staff  member 
stated  that  they  needed  more  training  on  the  function  and  use  of  the  digital  systems  and  on 
working  together  as  a  team.  The  additional  hands-on  “train-up”  time  recommended  by  battle 
staff  members  ranges  from  two  days  to  two  weeks.  None  of  the  battle  staff  members’  written 
comments  indicate  a  need  for  additional  training  on  the  MDMP  in  general  or  additional  training 
on  how  to  perform  a  specific  MDMP  step.  Similarly,  battle  staff  members’  written  comments 
shed  no  new  light  on  the  specific  problems  that  directly  resulted  from  inadequate  familiarity  with 
the  digital  systems  or  from  inadequate  prior  training.  The  written  comments  for  Items  10  and  14 
mirror  those  listed  above  in  the  subsection  that  discusses  mission  planning  errors  and  oversights. 

Defining  TTP.  There  are  many  Army  publications,  mostly  field  manuals,  which  purport 
to  document  TTPs  for  some  particular  organizational  unit  or  a  particular  activity.  TTP  is  a 
commonly  used  term  and  generally  assumed  to  have  an  intrinsic  and  commonly  understood 
meaning.  This  assumption  is  incorrect.  The  authors  have  been  unsuccessful  in  extracting  a 
formal  definition  of  TTP  from  Army  doctrine.  Nor  is  there  any  known  consistent  format  for 
documentation  of  TTP.  In  the  absence  of  a  definition  of  the  term  TTP  as  a  whole  it  is  logical  to 
seek  definitions  of  the  component  terms.  The  term  tactics  is  defined  by  the  Department  of 
Defense  (Joint  Chiefs  of  Staff,  1994)  as:  "1.  The  employment  of  units  in  combat.  2.  The 
ordered  arrangement  and  maneuver  of  units  in  relationship  to  each  other  and/ or  to  the  enemy  in 
order  to  fully  utilize  their  full  potentialities".  A  general  definition  of  tactics  (Webster's  New  ’ 
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World  Dictionary  of  the  American  Language)  is:  "the  science  of  arranging  and  maneuvering 
military  or  naval  forces  in  action  or  before  the  enemy,  esp.  (as  distinguished  from  STRATEGY) 
with  reference  to  short-range  objectives".  The  same  reference  defines  technique  as:  "the  method 
of  procedure  (with  reference  to  practical  or  formal  details),  or  way  of  using  basic  skills,  in 
rendering  an  artistic  work  or  carrying  out  a  scientific  or  mechanical  operation".  Technique  does 
not  appear  to  be  defined  in  military  publications.  Procedure  is  doctrinally  defined  (Joint  Chiefs 
of  Staff,  1994)  in  this  manner:  "A  procedure  begins  with  a  specific,  documentable  event  that 
causes  an  activity  to  occur.  The  activity  must  produce  a  product  that  normally  affects  another 
external  organization.  Frequently,  that  product  will  be  the  event  that  causes  another  procedure  to 
occur.  It  is  important  to  recognize  that  a  procedure  determines  'what'  an  organization  must  do  at 
critical  periods  but  does  not  direct  'how'  it  will  be  done."  The  general  definition  (Webster's  New 
World  Dictionary  of  the  American  Language)  is:  "the  act,  method  or  manner  of  proceeding  in 
some  process  or  course  of  action;  esp.,  the  sequence  of  steps  to  be  followed". 

Aside  from  the  remarkable  degree  of  circularity  of  this  last  definition,  there  appears  to  be 
considerable  overlap  in  the  meanings  of  the  three  component  terms.  This  leads  us  to  a  question 
as  to  whether  they  are  distinct  concepts  or  synonyms.  The  conglomerate  term,  tactics, 
techniques  and  procedures  provides  a  satisfying  degree  of  alliteration  and  meter  that  may 
contribute  to  its  common  use,  perhaps  to  refer  to  a  single  class  of  behavioral  prescriptions 
relating  to  the  execution  of  military  operations.  Under  an  assumption  that  the  component  terms 
are  distinct,  though  overlapping,  concepts  then  their  interrelationships  require  definition  in  order 
to  clarify  the  meaning  of  the  composite.  It  seems  evident  that  the  components  all  refer  to 
prescriptions  for  classes  of  action.  It  is  possible  that  these  classes  of  action  could  be  parallel, 
sequential  or  hierarchically  related.  This  last  relationship  appears  to  best  fit  the  use  of  the 
component  terms  and  the  applications  in  which  the  composite  term  is  employed.  It  is  also 
possible  that  there  are  two  coexisting  relationships  among  these  terms.  Two  terms  may  be 
parallel  concepts  that  contribute  to  one  higher  level  term. 

The  core  concept  of  the  component  term  tactics  appears  to  be  arrangement  (maneuvering 
may  be  considered  a  time-varying  extension  of  arrangement).  In  this  sense  it  is  more  distinct 
from  technique  and  procedure  than  those  two  are  from  each  other.  Tactics  appears  to  be  the  top- 
level  component,  to  which  the  others  contribute.  It  appears  to  involve  the  spatial  and  temporal 
relationships  of  the  major  functional  elements  of  mission  execution  (e.g.  two  scout  aircraft,  a 
company  of  attack  aircraft  and  a  TOC).  The  core  concept  of procedure  appears  to  be  a 
combination  of  manner  and  sequence.  Therefore  procedure  may  entail  specification  of,  for 
example,  mission  phases  and  tasks,  and  their  spatial  and  temporal  interrelationships.  The  core 
concept  of  technique  appears  to  be  a  combination  of  method  and  skill.  Therefore  technique  may 
refer,  for  example,  to  the  methods  by  which  control  procedures  are  ensured  or  communication 
transactions  are  completed. 

We  propose  that  the  interrelational  structure  of  the  components  of  the  term  TTP  is 
hierarchical  where  techniques  comprise  procedures  which  comprise  tactics,  despite  the  violence 
this  does  to  the  poetic  aspects  of  the  composite  term. 

Inadequate  TTPs.  Item  1 1  asked  battle  staff  members  to  indicate  whether  or  not  the 
TTPs  they  employed  were  inadequate  for  performing  the  MDMP  steps  and,  if  so,  to  identify  the 
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TTPs  that  were  inadequate.  For  every  MDMP  step,  at  least  42%  of  the  responses  indicate  that 
the  TTPs  were  considered  inadequate  for  performing  the  step.  The  percentages  of  inadequate 
TTP  responses  were  about  60%  for  the  steps  Develop  the  CO  A,  Analyze  the  CO  A,  and  Compare 
the  CO  A. 

The  percentages  of  inadequate  TTP  responses  range  widely  (from  0.0%  to  100%)  from 
one  battle  staff  position  to  another,  but  the  percentages  are  uniformly  high  for  the  battle  staff 
members  who  were  most  actively  involved  in  mission  planning  (XO  =  100%,  S2  =  100%,  S3  = 
76%,  S4  =  100%,  and  FSO  =  100%). 

The  battle  staff  members’  written  descriptions  of  inadequate  TTPs  provided  very  little 
useful  information  about  specific  TTPs  that  were  inadequate  or  about  new  TTPs  that  were 
needed.  Although  every  battle  staff  member  provided  a  written  response  for  Item  11,  the 
responses  describe  digital  system  design  deficiencies  or  training  deficiencies  rather  than  TTPs. 
The  reasons  why  battle  staff  members  described  problems  other  than  TTPs  are  not  known.  One 
possible  reason  is  that  battle  staff  members  received  no  training  on  TTPs  that  are  needed  for 
operating  effectively  in  a  digital  TOC.  This  explanation  is  supported  by  the  S2’s  comment  that 
he  could  not  assess  the  adequacy  of  existing  TTPs  because  he  had  no  idea  what  they  were.  A 
second  possible  reason  is  that  the  term  TTP  appears  to  have  been  poorly  understood  by  members 
of  the  battle  staff. 

All  of  the  problems  identified  in  the  battle  staff  members’  written  comments  for  Item  11 
have  been  mentioned  earlier  in  the  discussions  of  responses  to  other  questionnaire  items. 

Need  for  additional  automation.  Battle  staff  members  were  asked  to  indicate  if  more 
automation  is  needed  to  perform  an  MDMP  step  with  sufficient  accuracy,  comprehensiveness,  or 
speed.  For  the  seven  MDMP  steps,  between  24.0%  and  43.5%  of  the  responses  indicate  that 
more  automation  is  needed  to  perform  the  step.  However,  most  of  the  suggestions  for  additional 
automation  are  aimed  at  improving  performance  on  two  or  more  MDMP  steps.  Summarized 
below  are  the  recommendations  that  generated  most  of  the  responses  indicating  that  increased 
automation  is  needed. 

•  The  CO  identified  a  need  for  an  easy  way  to  make  annotations  on  digital  displays. 
Specifically,  the  CO  suggested  the  development  of  a  “John  Madden  pen”  that  would 
enable  him  to  quickly  sketch  his  vision  onto  the  MCS,  thereby,  providing  the  basis  for 
the  subsequent  development  of  the  COA  graphics. 

•  The  CO  suggested  that  a  system  should  be  developed  whereby  designating  (i.e., 
double  clicking)  an  enemy  vehicle  icon  will  cause  the  system  to  search  an  OPFOR 
database  and  to  display  information  about  the  vehicle. 

•  The  XO  and  the  S2  suggested  that  more  automation  is  needed  to  eliminate  the 
requirement  for  manual  entry  of  spot  reports  and  other  information  that  already  is  in 
digital  form  in  the  same  or  another  system.  In  particular,  it  should  not  be  necessary  to 
enter  manually  the  same  information  into  both  the  ASAS  and  the  MCS. 

•  The  XO,  the  S2,  and  the  S3  suggested  that  more  automation  is  needed  to  facilitate  the 
creation  and  transfer  of  graphics.  The  system  should  provide  for  the  automatic 
distribution  of  updated  graphics. 
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This  subsection  ends  with  comments  about  the  practical  limitations  of  using  observers  to 
gather  useful  information  about  mission  planning  problems,  especially  mission  planning  errors 
and  oversights. 

Time  spent  on  MDMP  steps.  The  mission  planning  observers  recorded  the  amount  of 
time  spent  on  each  of  six  MDMP  steps.  The  times  recorded  for  Mission  1  and  Mission  3  are 
shown  in  Table  5.  The  last  row  shows  the  total  time  spent  on  all  six  MDMP  steps.  The  right- 
hand  column  shows  the  mean  time  for  the  six  MDMP  steps,  individually  and  collectively. 

Considerably  more  time  was  spent  planning  Mission  1  (6.0  hours)  than  Mission  3  (3.7  hours). 
With  only  one  exception,  the  time  spent  on  the  individual  MDMP  steps  was  greater  for  Mssion 
1  than  for  Mission  2;  the  time  spent  evaluating  the  COA  was  23  minutes  longer  for  Mssion  3 
than  for  Mission  1 .  The  largest  difference  is  found  for  the  step  Analyze  the  Mission.  Over  two 
hours  were  spent  analyzing  Mission  1,  and  only  18  minutes  were  spent  analyzing  Mission  2.  For 
all  other  MDMP  steps,  the  difference  in  the  time  spent  accomplishing  the  step  ranges  from  10 
minutes  (Receive  the  Mission)  to  40  minutes  (Approve  the  COA). 

Because  of  the  experience  gained  in  planning  Mission  1  and  Mission  2,  the  efficiency  of  the 
planning  team  undoubtedly  was  greater  for  Mssion  3  than  for  Mission  1.  However,  increased 
efficiency  cannot  be  considered  the  only  cause  or  even  the  primary  cause  of  the  decline  in  the 
planning  time  from  Mission  1  to  Mssion  3.  The  differences  in  the  time  spent  on  the  MDMP 
steps  can  be  attributed  in  large  part  to  differences  in  the  types  of  missions15  that  were  being 
planned  and  differences  in  the  total  amount  of  time  available  for  mission  planning.  The 
confounding  of  experience  with  mission  type  make  it  impossible  to  assess  the  independent 
effects  of  increased  efficiency  and  mission  type  on  mission  planning  time. 

Table  5 

Mission  planning  times  recorded  by  the  mission  planning  observer. 


Planning  Time  (in  Minutes) 

MDMP  Step 

Mission  1 

Mission  3 

Mean 

Receive  the  Mission 

20 

10 

15 

Analyze  the  Mission 

130 

18 

74 

Develop  the  COA 

30 

60 

45 

Evaluate  the  COA 

60 

83 

71.5 

Compare  the  COA 

60 

30 

45 

Approve  the  COA 

60 

20 

40 

Total 

360 

221 

290.5 

The  total  planning  time  recorded  by  the  observer  was  considerably  less  than  the  total 
planning  times  reported  by  battle  staff  members  (see  Table  2).  Battle  staff  members  indicated 


15  It  will  be  recalled  that  the  purpose  of  the  first  mission  planning  session  was  to  plan  a  deliberate  attack,  and  the 
purpose  of  third  session  was  to  plan  a  hasty  attack. 
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that  at  least  10  hours  were  spent  planning  both  Mission  1  and  Mission  3,  which  is  considerably 
more  than  the  total  times  shown  in  Table  5  (6.0  hours  for  Mission  1  and  3.7  hours  for  Mission  3). 
One  reason  for  the  differences  is  that  Table  2  includes  the  time  required  to  Produce  the  Orders, 
but  Table  5  does  not.  The  step  Produce  the  Orders  was  inadvertently  left  off  of  the  observer’s 
data  collection  form. 

A  second  reason  for  the  time  differences  is  that  some  battle  staff  members  spent  time 
after  regular  duty  hours  on  mission  planning  activities.  The  after-hours  mission  planning 
activities  were  unscheduled  but  not  unexpected.  Nevertheless,  there  was  no  opportunity  to 
compile  information  about  the  battle  staff  members  who  engaged  in  after-hours  mission  planning 
or  the  MDMP  steps  on  which  they  worked.  The  information  gained  through  unstructured 
interviews  suggests  that  the  XO,  the  S2,  and  the  S3  engaged  in  after-hours  mission  planning; 
however,  no  attempt  was  made  to  question  them  about  the  work  they  performed  during  the  after- 
hours  mission  planning  sessions. 

A  third  reason  for  the  time  differences  is  that  battle  staff  members  reported  elapsed  time, 
and  the  observer  reported  only  the  time  spent  working.  The  observer  did  not  include  lunch 
breaks  or  other  breaks  taken  during  the  time  an  MDMP  step  was  being  performed.  In  addition, 
the  observer  did  not  include  the  time  that  transpired  between  the  completion  of  one  MDMP  step 
and  the  start  of  the  next  step.  For  all  of  the  above  reasons,  the  times  shown  in  Table  5  are 
accurate  estimates  of  the  time  that  the  planning  team  spent  working  on  the  individual  MDMP 
steps  that  are  listed.  However,  the  total  times  shown  in  Table  5  underestimate  the  time  spent 
planning  the  entire  mission,  because  it  does  not  include  the  time  spent  producing  the  orders. 

Battle  staff  members’  participation  in  MDMP  substeps.  Mission  planning  observers 
recorded  each  MDMP  substep  that  was  accomplished  and  recorded  the  battle  staff  members  who 
contributed  to  the  accomplishment  of  each  substep.  The  data  compiled  by  the  mission  planning 
observers  show  that  some  amount  of  effort  was  expended  on  most  of  the  MDMP  substeps  that 
are  specified  by  doctrine.  Some  amount  of  effort  was  spent  on  nearly  86%  of  all  substeps  during 
the  planning  of  Mission  1  and  on  about  75%  of  all  substeps  during  the  planning  of  Mission  3. 

Of  the  28  substeps  that  were  not  performed  when  planning  Mission  3,  21  are  substeps  for 
the  MDMP  step  Evaluate  the  COA.  For  all  other  MDMP  steps,  the  number  of  substeps  not 
performed  was  about  the  same  for  Mission  1  and  Mission  3.  Although  about  one-half  of  the 
substeps  were  performed  when  evaluating  the  COA  for  Mission  3,  battle  staff  members  spent 
more  time  evaluating  the  Mission  3  COA  (83  minutes)  than  they  spent  evaluating  the  Mission  1 
COA  (60  minutes).  These  findings  suggest  that  battle  staff  members  accommodated  to  the 
reduced  planning  time  available  for  Mission  3  by  (a)  eliminating  substeps  from  the  MDMP  step 
Evaluate  the  COA  and  (b)  reducing  the  amount  of  time  spent  on  all  MDMP  steps  except 
Evaluate  the  COA.  It  is  particularly  noteworthy  that  battle  staff  members  did  not  reduce  the 
amount  of  time  they  spent  Analyzing  the  COA  for  Mission  3  even  though  they  attempted  to 
perform  only  about  one-half  of  the  substeps  specified  for  that  MDMP  step. 

Another  interesting  difference  between  Mission  1  planning  and  Mission  3  planning  is  the 
number  of  battle  staff  members  who  worked  on  a  substep.  Figure  1 1  shows: 
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•  the  percentage  of  substeps  that  were  not  performed  (designated  none), 

•  the  percentage  of  substeps  that  were  accomplished  by  a  single  battle  staff  member 
(designated  I), 

•  the  percentage  of  substeps  for  which  two  battle  staff  members  contributed  (designated 
2),  and 

•  the  percentage  of  substeps  for  which  three  or  more  battle  staff  members  contributed 
(designated  3  or  more). 

It  is  clear  from  Figure  1 1  that  battle  staff  members  used  a  different  strategy  on  Mission  1  than 
Mission  3,  presumably  because  less  time  was  available  for  planning  Mission  3  than  Mission  1. 

A  significantly  larger  number  of  substeps  were  performed  by  one  battle  staff  member  during 
Mission  1  planning  than  during  Mission  3  planning,  %2  (1,  101)  =  7.2 ,p<  .01.  In  contrast, 
significantly  fewer  substeps  were  performed  by  two  staff  members  (as  a  team)  during  Mission  1 
planning  than  during  Mission  3  planning,  %2  (1,  60)  =  B.\,p<  .01.  Although  fewer  substeps 
were  performed  during  the  planning  of  Mission  3  (12  fewer),  the  difference  was  not  quite  large 
enough  to  reach  the  .05  level  of  confidence,  %2  (1,44)  =  3.3,  p>  .10.  The  two  missions  do  not 
differ  in  the  number  of  substeps  that  were  performed  by  a  team  of  three  or  more  battle  staff 
members,  %2  (1,  19)  =  2.6,  p>  .20. 

These  findings  indicate  that  the  strategy  employed  to  compress  mission  planning  time  for 
Mission  3  consisted  of  (a)  eliminating  some  substeps  (mainly  for  the  MDMP  step  Evaluate  the 
CO  A),  (b)  performing  the  sub  steps  of  other  MDMP  steps  in  less  time,  and  (c)  performance  of 
more  substeps  as  a  two-person  team  and  fewer  substeps  by  single  individuals.  It  was  not 
possible  to  question  battle  staff  members  to  determine  whether  or  not  the  strategy  used  to 
compress  planning  time  was  a  deliberate,  conscious  strategy.  Moreover,  no  data  are  available  to 
determine  whether  the  strategy  employed  is  more  or  less  effective  than  alternate  strategies. 
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Figure  11.  Percentage  of  all  MDMP  substeps  that  were  not  performed  and  the  percentage  of 
substeps  performed  by  one,  two,  or  three  or  more  battle  staff  members. 
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The  mission  planning  observations  also  provided  data  on  the  extent  to  which  individual 
battle  staff  members  performed  the  substeps  assigned  to  their  battle  staff  position  by  the  MDMP 
doctrine  (see  FM  100-5).  Appendix  F  contains  a  complete  listing  of  the  substeps  performed  by 
each  battle  staff  member  for  each  mission.  The  data  for  Mission  1  and  Mission  3  are 
summarized  in  Figure  12  and  Figure  13,  respectively.  In  both  Figure  12  and  Figure  13,  a  bar 
with  three  differently  shaded  portions  is  shown  for  each  of  seven  battle  staff  positions.  The 
meaning  of  each  of  the  three  portions  of  the  bars  is  explained  below. 

•  The  black  portion  of  a  bar  represents  the  number  of  substeps  assigned  to  the  battle 
staff  position  (by  MDMP  doctrine)  that  were  actually  performed  by  the  individual 
assigned  to  that  battle  staff  position.  The  legend  title  for  assigned  sub  steps  that  were 
performed  is  Yes  (doctrine). 

•  The  white  portion  of  the  bar  shows  the  number  of  substeps  that  were  performed  by 
the  battle  staff  member  that  was  not  assigned  to  the  corresponding  battle  staff  position 
(by  the  MDMP  doctrine).  The  legend  title  for  these  substeps  that  were  performed  by 
not  assigned  is  Yes  (other). 

•  The  stippled  portion  of  the  bar  shows  the  number  of  substeps  assigned  to  the  battle 
staff  position  (by  MDMP  doctrine)  that  were  not  performed  by  the  battle  staff 
member  who  occupied  that  position.  The  legend  title  for  substeps  that  were  assigned 
but  not  performed  is  No  (Doctrine). 
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Figure  12.  Distribution  of  MDMP  substeps  performed  and  MDMP  substeps  not  performed  by 
battle  staff  members  during  the  planning  of  Mission  1. 

When  interpreting  the  data  shown  in  Figure  12  and  Figure  13,  it  should  be  kept  in  mind 
that  the  number  of  substeps  assigned  by  MDMP  doctrine  is  not  the  same  for  all  battle  staff 
positions.  The  MDMP  doctrine  assigns  the  largest  number  of  substeps  to  the  S3  (59  substeps) 
and  the  S2  (35  substeps).  About  25  substeps  are  assigned  to  each  of  the  remaining  battle  staff 
positions. 


mmtmA 


■  Yes  (Doctrine) 

□  Yes  (Other) 

□No  (Doctrine) 

56 


A  comparison  of  the  distributions  shown  in  Figure  12  and  Figure  13  reveals  useful 
insights  about  the  battle  staffs  strategy  for  accommodating  to  the  compressed  planning  time  for 
Mission  3.  The  most  striking  change  is  in  the  distribution  of  substeps  among  the  battle  staff 
members.  In  planning  Mission  1,  every  battle  staff  member  performed  or  assisted  in  performing 
13  or  more  substeps  (see  Figure  12).  The  largest  number  of  substeps  were  performed  by  the  S3 
(44  substeps),  the  XO  (32  substeps),  and  the  S2  (29  substeps).  The  substeps  performed  in 
planning  Mission  3  (see  Figure  13)  were  far  less  evenly  distributed  among  battle  staff  members 
than  in  planning  Mission  1.  The  CO,  SI,  S2,  S4,  and  FSO  performed  fewer  substeps  in  planning 
Mission  3  than  in  planning  Mission  1.  Conversely,  both  the  XO  and  S3  performed  a  larger 
number  of  steps  and  a  larger  percentage  of  substeps  in  planning  Mission  3  than  in  planning 
Mission  1. 


Number  of  MDMP  Substeps 


Figure  13.  Distribution  of  MDMP  sub  steps  performed  and  MDMP  not  performed  by  battle  staff 
members  during  the  planning  of  Mission  3. 

Together,  the  XO  and  S3  performed  43.7%  of  the  141  substeps  that  were  performed  in  planning 
Mission  1  and  74.5%  of  the  141  substeps  that  were  performed  in  planning  Mission  3.  After 
normalizing  the  frequencies  for  the  same  total  number  of  substeps  performed,  a  %2  test  shows 
that  the  number  of  substeps  performed  by  the  XO  and  S3  (combined)  was  significantly  smaller 
for  Mission  1  than  Mission  3,  %2  (2,  181)  =  4.3,  p<  .05. 

Another  interesting  difference  between  Mission  1  and  Mission  3  is  the  number  of 
nondoctrinal  substeps  performed  by  the  battle  staff  members.  In  planning  Mission  1,  the  S3  and 
the  S4  performed  3  nondoctrinal  steps,  the  CO  performed  6  nondoctrinal  steps,  the  SI  and  S2 
performed  9  nondoctrinal  steps,  and  the  XO  performed  16  nondoctrinal  steps.  In  planning 
Mission  3,  the  number  of  nondoctrinal  steps  performed  decreased  for  every  member  of  the  battle 
staff  except  for  the  XO  and  S3.  From  Mission  1  to  Mission  3,  the  number  of  nondoctrinal 
substeps  performed  by  the  XO  and  S3  increased  from  16  to  36  and  from  3  to  22,  respectively.. 
Together,  the  XO  and  S3  performed  a  significantly  larger  number  of  nondoctrinal  tasks  during 
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the  planning  of  Mission  1  (19  substeps)  than  during  the  planning  of  Mission  3  (58  substeps),  x2 
(1,  91)  =  17.2,  p<  .001. 

Other  observations.  Items  on  the  data  collection  forms  asked  observers  to  record  other 
events  that  occurred  during  mission  planning,  particularly  the  types  of  problems  that  battle  staff 
members  encountered  when  attempting  to  perform  the  MDMP  steps.  For  each  MDMP  step, 
observers  were  asked  to  record  each  instance  of  a  variety  of  specific  problems  listed  on  the  data 
collection  form  (see  Appendix  C).  For  the  reasons  discussed  in  the  next  subsection,  observers 
were  limited  in  their  ability  to  detect  and  report  many  of  the  types  of  problems  that  were  of 
interest.  Even  so,  a  number  of  useful  observations  were  recorded  by  the  mission  planning 
observers.  The  following  general  observations  were  derived  by  synthesizing  the  specific 
comments  that  observers  recorded  during  the  three  mission  planning  sessions. 

•  Distribution  of  OPORD 

-  Some  inefficiency  resulted  from  the  failure  to  distribute  the  OPORD  to 
all  the  staff  members  who  needed  it.  This  problem  occurred  in 
distributing  the  OPORD  for  Mission  1  and,  to  a  lesser  extent,  the  OPORD 
for  Mission  3. 

•  Use  of  Digital  Equipment 

-  At  the  outset,  all  members  of  the  battle  staff  lacked  knowledge  about  the 
capabilities  of  the  AS  AS  and  MCS.  Without  the  civilian  operators, 
neither  the  AS  AS  nor  MCS  could  have  been  used.  The  S2  had  difficulty 
accessing  information  available  in  the  MCS  and  ASAS  and  had  difficulty 
using  the  ASAS  to  produce  threat  overlays.  Staff  members  encountered 
problems  when  attempting  to  export  graphics  (i.e.,  overlays)  to  another 
digital  system. 

-  The  digital  equipment  was  used  very  little  in  the  planning  of  Mission  1. 
However,  all  members  of  the  battle  staff  were  using  the  digital  equipment 
much  more  extensively  and  much  more  effectively  by  the  time  they 
commenced  planning  Mission  3. 

-  When  planning  Mission  3,  the  digital  equipment  was  used  much  more 
effectively  in  performing  terrain  analysis  and  in  wargaming.  However, 
when  planning  Mission  3  the  battle  staff  still  commenced  the  planning 
process  with  an  analog  map  analysis  and  butcher  paper  outlines  of  CO  As. 

•  Workload 

-  Many  instances  were  observed  in  which  battle  staff  member’s  workload 
was  very  heavy.  However,  no  evidence  of  excessive  workload  was 
observed. 

•  Need  for  Additional  Automation 

-  No  evidence  of  a  need  for  additional  automation  was  observed. 

•  Need  for  Information  Manager 

-  There  is  a  need  for  a  staff  member  who  is  responsible  for  entering  analog 
information  into  the  ASAS  and  MCS. 

Limitations  of  mission  plannin2  observations.  The  mission  planning  observers  did  an 
excellent  job  observing  and  recording  the  MDMP  substeps  that  were  accomplished,  the  battle 
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staff  members  who  participated  in  each  MDMP  substep,  and  the  time  spent  on  each  MDMP  step. 
However,  they  found  it  difficult  to  observe  how  a  battle  staff  member  accomplished  an  MDMP 
substep  or  how  well  the  substep  was  performed.  The  mission  planning  observers’  difficulties 
resulted  from  three  factors.  First,  a  large  portion  of  mission  planning  activities  is  cognitive  and 
not  directly  observable.  The  only  way  to  obtain  information  about  cognitive  activities  is  to 
question  battle  staff  members  frequently  about  what  they  are  thinking.  Unfortunately,  frequent 
queries  about  cognitive  processes  tend  to  be  so  obtrusive  that  they  modify  the  planning  process 
in  important  ways.  Second,  the  large  number  of  battle  staff  members  and  their  geographical 
dispersion  make  it  difficult  to  continuously  observe  every  staff  member  with  a  reasonable 
number  of  observers.  Third,  a  great  deal  of  experience  in  mission  planning  is  needed  to  detect 
battle  staff  members’  errors  and  oversights  by  observing  the  products  of  the  various  MDMP 
steps.  It  is  difficult  to  locate  individuals  with  the  requisite  experience  and,  if  located,  to 
successfully  solicit  their  support  for  the  amount  of  time  needed. 

These  findings  support  the  conclusion  that  the  use  of  mission  planning  observers  is  not  an 
effective  method  for  detecting  specific  types  of  problems  encountered  during  mission  planning, 
particularly  the  specific  errors  and  oversights  made  in  performing  specific  MDMP  steps  and 
substeps. 

Mission  Execution 


This  subsection  of  the  report  describes  the  data  that  were  compiled  through  observations 
of  the  events  that  occurred  in  the  battalion  current  operations  cell.  As  discussed  earlier,  the 
observation  procedures  were  developed  mainly  to  provide  useful  data  about  the  flow  and 
management  of  information  during  mission  execution.  It  was  assumed  that  such  data  could  be 
used  to  draw  valid  inferences  about  a  variety  of  important  issues,  including  workload  levels,  the 
CO’s  situation  awareness,  information  management  procedures,  and  the  effectiveness  with  which 
the  digital  systems  are  used. 

The  observation  procedures  also  were  designed  to  record  information  about  problems  that 
members  of  the  battle  staff  encountered  in  performing  their  jobs  during  mission  execution.  It 
was  assumed  that  the  knowledge  of  such  problems  could  be  used  to  identify  the  need  for 
improved  (a)  TTPs,  (b)  training,  and  (c)  research  methodology. 

The  observation  procedures  were  developed  to  compile  data  on  the  use  of  each  of  the 
digital  systems.  Unfortunately,  only  the  AS  AS,  the  MCS,  and  the  CSSCS  were' consistently 
operational  during  the  period  of  this  study.  Although  it  was  not  possible  to  observe  the  use  and 
utility  of  the  other  digital  systems,  the  observations  provided  useful  baseline  data  on  the  flow  and 
management  of  information  when  the  AMDW/S,  the  FAADEO,  and  the  AFATDS  are  not 
available  for  use  during  mission  execution. 

Tabulation  of  Voice  Communication  Events 

Voice  communications  were  tabulated  in  a  variety  of  ways  in  an  attempt  to  identify  the 
tabulation  method  that  best  meets  the  needs  of  this  study.  The  tabulation  method  that  was 
decided  upon  is  described  briefly  below. 
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A  “communication  event”  was  established  as  the  basic  unit  of  analysis.  A 
communication  event  is  defined  here  as  a  single  one-way  transmission  of  information  from  one 
individual  to  another.  For  example,  a  question  transmitted  from  person  A  to  person  B  was 
counted  as  one  communication  event  and  the  answer  transmitted  from  person  B  back  to  person  A 
was  counted  as  a  second  communication  event.  Because  communication  events  tended  to  be 
uniformly  short  in  duration,  the  number  of  communication  events  per  unit  time  provides  a 
meaningful  metric  for  quantifying  the  volume  of  voice  communication  for  battle  staff  members, 
individually  and  collectively. 

All  the  communication  events  that  were  recorded  in  the  battalion  current  operations  cell 
were  classified  into  one  of  three  categories  listed  below. 

•  Category  A:  communication  events  that  involved  the  CO 

•  Category  B:  communication  events  that  involved  the  XO  but  not  the  CO 

•  Category  C:  communication  events  that  involved  another  battle  staff  member  but  not 
the  CO  or  XO 

It  is  important  to  emphasize  that  this  procedure  resulted  in  the  classification  of 
communication  events  into  mutually  exclusive  and  comprehensive  categories.  A 
communication  event  that  involved  both  the  CO  and  XO  was  classified  only  into  Category  A.  A 
communication  event  that  involved  the  XO  and  the  S3  was  classified  only  into  Category  B.  A 
communication  event  that  involved  any  battle  staff  member  other  than  the  CO  and  XO  was 
classified  only  into  Category  C.  Because  communication  events  were  classified  into  mutually 
exclusive  categories,  the  sum  of  the  communication  events  in  the  three  categories  is  equal  to  the 
total  number  of  communication  events  that  were  recorded. 

The  communication  events  in  each  of  the  three  categories  were  further  subdivided  with 
the  respect  to  the  individual  who  originated  the  communication  event  and  the  recipient  of  the 
initial  transmission.  The  results  of  the  classification  and  tabulation  of  communication  events  are 
described  in  the  following  subsections. 

Volume  of  Voice  Communication  Events 

The  volume  of  voice  communication  events  is  defined  here  as  the  number  of 
communication  events  that  occurred  during  a  15-minute  period.  Communication  event  volume 
can  be  used  to  draw  useful  inferences  about  the  workload  of  the  battle  staff  members, 
individually  and  collectively.  It  is  important  to  point  out  that  a  low  volume  of  voice 
communication  events  does  not  necessarily  mean  that  workload  is  low.  Even  if  the  volume  of 
voice  communication  events  is  low,  battle  staff  members  could  be  engaged  in  a  host  of  other 
activities  that  impose  a  high  level  of  cognitive  workload  on  them.16  Conversely,  it  is  safe  to  infer 
a  high  level  of  workload  when  the  volume  of  voice  communication  events  is  high.  Because 
humans  are  limited  in  their  capability  to  perform  two  tasks  concurrently,  they  have  little  excess 
attentional  capacity  when  they  are  engaged  in  voice  communication. 


16  A  reduction  in  the  volume  of  voice  communication  is  sometimes  an  indication  of  high  workload.  When  engaged 
in  a  task  that  has  higher  priority  than  voice  communication,  all  voice  communication  may  be  delayed  until  the  ' 
higher  priority  task  has  been  completed. 
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The  four  charts  in  Figure  14  show  the  number  of  communication  events  per  15-minute 
interval  that  occurred  in  the  battalion  current  operations  cell  throughout  Day  1,  Day  2,  Day  4, 
and  Day  5  (Day  3  activities  did  not  include  any  operations).  The  charts  are  “stacked”  line  charts. 
The  number  of  communication  events  involving  the  XO  (shaded  area)  are  “stacked”  on  the 
number  of  communication  events  involving  the  CO  (black  area).  Similarly,  the  number  of 
communication  events  involving  other  battle  staff  members  (white  area)  are  “stacked”  on  the 
communication  events  involving  the  XO  (shaded  area)  and  those  that  involved  the  CO  (black 
area).  The  upper-most  line  in  each  chart  shows  the  total  number  of  communication  events  per 
15-minute  interval.  For  example,  the  fourth  period  on  Day  1  shows  no  events  for  the  CO,  7 
events  for  the  XO  and  10  events  for  all  others;  a  total  of  17  events  observed.  The 
communication  events  shown  in  Figure  14  are  conservative  estimates  of  the  number  of 
communication  events  that  actually  occurred.  Observers  undoubtedly  missed  some 
communication  events  during  the  periods  when  the  volume  of  communication  was  very  high. 
However,  because  of  the  procedures  that  were  used,  there  is  no  chance  that  observers  would  have 
recorded  communication  events  that  did  not  occur. 

Figure  14  shows  that  both  the  volume  and  the  distribution  of  communication  events 
varied  widely  during  the  day  and  from  one  day  to  another.  The  intra-day  variability  in  the 
volume  of  communication  events  is  due  mainly  to  the  type  and  phase  of  the  mission  being 
conducted.  As  would  be  expected,  the  volume  increased  abruptly  at  about  the  time  aircraft 
and/or  UAV  were  launched,  remained  high  for  at  least  one  hour,  and  decreased  abruptly  at  about 
the  time  aircraft  completed  their  egress  from  the  combat  areas.  The  volume  of  communication 
events  was  relatively  low  during  the  resupply  missions  and  reconnaissance  missions  that  were 
conducted  during  Day  1  and  the  morning  of  Day  2.  The  volume  was  much  higher  during  the 
attack  missions  flown  during  the  afternoon  of  Day  2  and  during  both  the  morning  and  afternoon 
of  Day  4  and  Day  5.  The  following  are  examples  of  observations  that  can  be  gleaned  from  a 
study  of  the  intra-  and  inter-day  variability  in  the  volume  of  communication  events  shown  in 
Figure  14. 


•  During  the  morning  of  Day  1,  the  XO  was  involved  in  a  substantial  number  of 
communication  events,  independent  of  the  CO,  and  the  other  battle  staff  members 
were  involved  in  a  substantial  number  of  communication  events  independent  of  the 
CO  and  XO.  The  communication  load  was  distributed  relatively  evenly  among  the 
battle  staff  during  the  morning  of  Day  1. 

•  During  the  afternoon  of  Day  2,  the  XO  was  involved  in  a  disproportionately  large 
number  of  communication  events  independent  of  the  CO. 

•  During  Day  4,  there  were  relatively  few  communication  events  in  which  the  CO  was 
not  involved.  (The  XO  was  not  present  in  the  current  operations  cell  during  Day  4.) 

•  During  the  first  part  of  the  mission  conducted  the  morning  of  Day  5,  the 
communication  load  was  distributed  fairly  evenly  among  the  CO,  the  XO,  and  the 
other  battle  staff  members.  During  the  last  45  minutes  of  the  mission,  however,  the 
CO  was  involved  in  nearly  every  communication  event. 
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•  The  heavy  involvement  by  the  CO  continued  during  the  mission  conducted  on  the 
afternoon  of  Day  5.  During  this  mission,  the  CO  was  involved  in  more  than  62%  of 
all  the  communication  events  that  occurred. 


Start  End 

Elapsed  Time  (15-Minute  Intervals) 


Figure  14.  Number  of  communication  events  per  15-minute  interval  that  involved  the  CO,  the 
XO,  and  all  other  members  of  the  battle  staff  on  Day  1,  Day  2,  Day  4,  and  Day  5. 
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The  total  volume  of  communication  events  (upper-most  line  in  Figure  14)  reached  higher 
levels  on  the  last  two  days  than  on  the  first  two  days.  The  total  volume  of  communication  events 
peaked  at  35  and  39  on  Day  1  and  Day  3,  respectively.  The  total  volumes  of  communication 
events  per  15  minute  interval  peaked  at  62  and  66  on  Day  4  and  Day  5,  respectively.  The 
temporal  pattern  of  the  total  volume  of  communication  events  differs  widely  from  one  mission  to 
another.  For  example,  note  the  large  difference  between  the  pattern  of  communication  events  for 
the  missions  conducted  during  the  morning  and  afternoon  of  Day  4.  During  the  morning  of  Day 
4,  the  total  volume  peaks  during  a  single  period  at  a  value  (51  events)  that  is  considerably  higher 
than  the  value  for  the  preceding  period  (28  events)  or  following  period  (27  events).  During  the 
afternoon  of  Day  4,  the  total  volume  peaks  at  a  value  of  62  events  and  remains  high  throughout 
the  two  following  periods  (55  events  and  61  events)  before  declining  to  a  value  of  27  events. 

The  mission  conducted  during  the  morning  of  Day  4  was  a  rehearsal  of  the  mission  conducted 
during  the  afternoon  of  Day  4,  so  the  differences  in  the  temporal  patterns  of  total  volume  are 
likely  due  to  differences  in  the  information  management  strategy  employed  by  the  battle  staff 
rather  than  to  differences  in  the  mission. 17 

An  examination  of  the  communication  events  for  the  CO  and  the  XO  reveals  that  their 
strategy  for  handling  voice  communication  varied  from  day  to  day.  On  Day  1,  it  appears  that  the 
responsibility  for  voice  communication  periodically  shifted  between  the  CO  and  XO.  During 
periods  in  which  the  number  of  communication  events  was  high  for  the  CO,  they  tended  to  be 
low  for  the  XO,  and  vice  versa.  No  shifting  of  responsibility  is  evident  in  the  data  for  the 
remaining  three  days.  On  Day  2,  the  volume  of  communication  events  was  substantially  higher 
for  the  XO  than  for  the  CO  during  the  only  high-volume  period  that  occurred  that  day.  However, 
the  volume  of  communication  events  was  clearly  higher  for  the  CO  during  every  high-volume 
period  that  occurred  during  Day  4  and  Day  5. 

There  were  a  substantial  number  of  time  periods  during  which  voice  communication 
imposed  a  very  high  level  of  workload  on  the  individual  (CO  or  XO)  who  was  mainly 
responsibility  for  voice  communication.  During  the  afternoon  of  Day  3,  for  example,  there  were 
three  15-minute  periods  during  which  the  CO  was  involved  in  between  33  and  45 
communication  events.  It  is  safe  to  conclude  that  the  CO  had  little  time  or  attentional  resources 
to  perform  other  tasks  when,  on  average,  he  was  involved  in  between  two  and  three 
communication  events  per  minute. 

The  absence  of  the  XO  is  one  reason  why  the  CO’s  volume  of  communication  events  was 
so  high  during  the  mission  conducted  the  afternoon  of  Day  4.  However,  the  CO’s  volume  of 
communication  events  was  very  high  during  the  mission  conducted  the  afternoon  of  Day  5  even 
though  the  XO  was  present  and  was  involved  in  a  substantial  number  of  communication  events. 
During  one  15-minute  period,  for  example,  the  CO  participated  in  43  communication  events,  and 
the  XO  participated  in  1 1  communication  events  that  did  not  involve  the  CO.  During  the  next 
period  the  numbers  of  communication  events  for  the  CO  and  XO  were  44  and  1 1,  respectively. 


17  The  mission  conducted  the  morning  of  Day  5  was  a  rehearsal  of  the  mission  conducted  in  the  afternoon.  So,  like 
Day  4,  the  differences  between  the  two  Day  5  missions  may  be  attributed  to  differences  in  the  battle  staffs’ 
information  management  strategies  rather  than  to  differences  in  mission  type. 
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The  numbers  in  the  “other”  category  can  be  correctly  interpreted  only  by  taking  into 
account  the  frequency  with  which  one  of  the  “other”  battle  staff  members  communicated  with 
the  CO  and/or  the  XO.  Data  are  presented  in  following  sections  that  show  that  there  were 
relatively  few  communication  events  in  which  one  of  the  “other”  battle  staff  members 
communicated  with  either  the  CO  or  the  XO.  For  this  reason,  it  is  safe  to  conclude  from  Figure 
14  that  the  relatively  small  volumes  of  communication  events  for  the  “other”  category  indicate 
that  voice  communication  did  not  impose  a  high  level  of  workload  on  the  “other”  member  of  the 
battle  staff,  individually  or  collectively. 

The  relationship  between  the  volume  of  communication  events  in  the  “CO”  category  and 
the  “other”  category  is  a  potentially  useful  indicator  of  the  information  management  strategy 
employed  by  a  battle  staff.  A  strong,  positive  relationship  (volumes  tend  to  rise  and  fall 
together)  indicates  that  the  communication  load  is  distributed  between  the  CO  and  the  other 
battle  staff  members.  A  negligible  or  a  negative  relationship  indicates  a  lack  of  distribution  of 
communication  load.  Pearson  product  moment  correlation  coefficients  (Guilford  and  Fructer, 
1978)  were  computed  to  quantify  the  relationship  between  the  volume  of  communication  events 
in  the  “CO”  category  and  the  “other”  category.  Correlation  coefficients  are  shown  in  Table  6  for 
the  communication  events  that  occurred  in  the  morning  and  in  the  afternoon  of  Day  1,  Day  2, 
Day  4,  and  Day  5. 

Table  6 

Pearson  product  moment  correlation  coefficients  between  the  volume  of  communication  events 
in  the  “CO”  category  and  the  volume  of  communication  events  in  the  “other”  category. 


Day 

Time  of 
Day 

Number 
of  Periods 

Pearson  r 

£ 

Day  1 

a.m. 

16 

.203 

>.10 

p.m. 

16 

.202 

>.10 

Day  2 

a.m. 

12 

-.356 

>.10 

p.m. 

14 

.322 

>.10 

Day  4 

a.m. 

12 

.735 

<.01 

p.m. 

7 

.581 

>.10 

Day  5 

a.m. 

13 

.207 

>.10 

p.m. 

8 

.166 

>.10 

There  was  only  one  time  period  (morning  of  Day  4)  for  which  the  volume  of 
communication  events  in  the  “CO”  category  is  correlated  significantly  with  the  volume  of 
communication  events  in  the  “other”  category,  r  (10)  =  .735,  p<  .01.  For  all  other  time  periods, 
the  correlation  coefficients  are  not  large  enough  to  reach  the  .05  level  of  confidence.  These 
findings  indicate  that  a  constant  information  management  strategy  was  not  used  by  the  battle 
staff.  More  important,  the  findings  indicate  that  the  strategy  employed  during  most  of  the  time 
periods  did  not  promote  the  distribution  of  communication  load  among  the  battle  staff  members. 
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Another  potentially  useful  indication  of  the  information  management  strategy  is  the 
relationship  between  (a)  the  total  volume  of  communication  events  and  (b)  the  percentage  of 
communication  events  in  which  the  CO  was  involved.  Figure  15  illustrates  the  nature  of  this 
relationship  for  four  15-minute  periods  that  occurred  the  afternoon  of  Day  5. 
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Figure  15.  Percentages  of  total  communication  events  in  which  the  CO,  the  XO,  and  other  battle 
staff  members  were  involved.  The  data  are  for  the  mission  conducted  during  the  afternoon  of 
Day  5. 

The  data  in  Figure  15  show  that  the  percentage  of  communication  events  in  which  other 
staff  members  were  involved  did  not  tend  to  increase  with  increases  in  the  total  volume  of 
communication  events.  During  the  first  period,  for  example,  “other”  staff  members  were 
involved  in  45.2%  of  the  3 1  communication  events  that  occurred.  During  the  following  three 
periods,  the  total  volume  of  events  is  substantially  higher  (between  54  and  66)  and  yet  the 
percentage  of  events  in  the  “other”  category  declines  first  to  33.3%,  then  to  16.7%,  and  finally  to 
a  low  value  of  6.8%.  The  data  for  the  first  four  periods  shown  in  Figure  15  show  an  inverse 
relationship  between  the  volume  in  the  “other”  category  and  the  total  volume  of  communication 
events.  However,  this  trend  (inverse  relationship)  changes  suddenly  in  the  fifth  period  in  which 
the  total  volume  declines  to  23  communication  events  and  the  percentage  in  the  “other”  category 
increases  to  47.8%. 

The  data  presented  above  suggest  that  analysis  of  the  volume  of  voice  communication 
events  provides  useful  information  about  both  the  absolute  and  the  relative  communication  load 
of  the  battle  staff  members.  The  data  also  suggest  that  an  analysis  of  changes  in  the  volume  of 
communication  for  the  different  battle  staff  members  provides  useful  information  about  changes 
in  the  information  management  strategies  that  are  employed  by  members  of  the  battle  staff.  Such 
changes  may  reflect  a  more  or  less  effective  distribution  of  the  voice  communication  load  among 
battle  staff  members.  Such  changes  also  may  reflect  a  more  or  less  effective  use  of  the  digital 
systems  for  transmitting  and  receiving  information.  That  is,  an  increase  in  the  use  of  the  digital 
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systems  to  convey  information  should  be  accompanied  by  a  corresponding  decline  in  the  volume 
of  voice  communication  events. 

Message  Content 

Each  voice  communication  event  was  classified  in  terms  of  the  content  of  the  message 
that  was  conveyed.  After  several  iterations,  14  message  classes  were  defined,  and  each 
communication  event  was  classified  into  the  most  suitable  class.  In  the  present  case,  multiple 
exchanges  about  the  same  topic  or  event  were  counted  as  only  one  message.  For  example,  a 
question  and  the  answer  to  that  question  were  counted  as  a  single  message.  Similarly,  a  request 
for  information  and  the  response  to  the  request  were  counted  as  a  single  message.  Table  7  shows 
the  percentage  of  messages  that  were  classified  into  each  of  the  14  message  classes. 

Table  7 

Percentage  of  daily  communication  events  that  were  classified  into  each  of  14  message  classes. 


Message  Category 

Day  1 
(March  2) 
k=351 

Day  2 
(March  3) 
n= 288 

Day  4 
(March  5) 
#*=383 

Day  5 
(March  6) 
m=487 

Total 

m=1509 

Orders/Requests  to  Take  Action 

8.0% 

11.1% 

22.7% 

25.9% 

18.1% 

Spot  Reports 

4.0% 

13.2% 

25.1% 

22.6% 

17.1% 

Messages  About  Mission  Plan/Status 

22.2% 

11.5% 

10.2% 

9.0% 

12.9% 

Messages  About  Aircraft  Location 

8.3% 

10.1% 

15.9% 

15.2% 

12.8% 

Intelligence  Other  Than  Spot  Reports 

7.7% 

8.0% 

4.4% 

7.4% 

6.8% 

Messages  About  OPORD/FRAGO/WO 

12.0% 

12.1% 

2.1% 

1.0% 

6.0% 

Messages  About  Downed  Aircraft 

9.7% 

6.6% 

0.3% 

3.7% 

4.8% 

BDA  Reports 

0.9% 

3.5% 

8.6% 

4.1% 

4.4% 

Questions  About  Digital  Systems 

7.7% 

4.9% 

1.8% 

2.3% 

3.9% 

Messages  About  CSS 

4.8% 

0.6% 

0.8% 

0.4% 

2.9% 

Requests  to  Change  Display 

3.1% 

3.5% 

1.3% 

3.5% 

2.8% 

Descriptions  of  Tasks  Performed 

3.1% 

1.4% 

1.3% 

1.6% 

1.9% 

Messages  About  Unit  Organization 

4.0% 

2.4% 

1.8% 

0.0% 

1.9% 

Other 

4.6% 

4.2% 

3.7% 

3.3% 

3.8% 

Table  7  shows  the  percentages  of  total  messages  that  were  classified  into  each  message 
class.  Percentages  are  shown  for  each  day  (Day  1,  Day  2,  Day  4,  and  Day  5)  and  for  the  total 
messages  pooled  across  the  four  days  (right-hand  column).  The  number  shown  at  the  top  of  each 
column  (e.g.,  n  =  35 1)  is  the  total  number  on  which  the  percentages  in  that  column  are  based. 

The  frequencies  for  any  cell  can  be  determined  by  multiplying  the  number  at  the  top  of  the 
column  by  the  percentage  value  in  the  cell  of  interest  and  dividing  the  product  by  100. 

The  data  in  Table  7  provide  useful  information  about  how  and  why  attentional  resources 
are  spent  conveying  information  among  battlefield  elements.  When  interpreting  these  data,  it 
should  be  kept  in  mind  that  a  very  large  percentage  of  the  messages  involve  the  CO  or  XO.  For 
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this  reason,  the  data  in  Table  7  are  heavily  influenced  by  the  information  needs  of  the  CO  and 
XO  and  by  their  need  to  convey  information  (including  orders)  to  other  elements.  For  the  most 
part,  the  meaning  of  the  data  presented  in  Table  7  is  self-evident.  Noteworthy  findings  that  may 
not  be  immediately  apparent  to  the  reader  are  described  briefly  below. 

Orders/requests  to  take  action.  Most  of  the  messages  in  this  class  are  orders  issued  by 
the  CO  or  XO  to  aircraft  pilots  or  to  the  UAV  operator  (e.g.,  change  your  position,  remain  in 
your  present  position,  attack  a  target,  do  not  attack  yet).  A  small  number  of  messages  in  this 
class  were  orders  or  requests  from  the  Brigade  to  the  CO  or  to  the  XO.  Orders  or  requests  to 
take  action  account  for  a  large  percentage  (18.1%)  of  all  messages  that  occurred  during  the  4 
days  for  which  missions  were  conducted.  However,  the  percentage  of  messages  in  this  class 
increases  dramatically  over  the  4-day  period.  The  percentage  increases  from  a  low  of  8.0%  on 
Day  1  to  a  high  of  25.9%  on  Day  5,  a  highly  significant  change,  x2  (1, 487)  =  46.1,  p<  .001.  The 
number  of  messages  in  this  class  is  partly  a  function  of  the  type  of  missions  that  were  conducted 
during  the  day.  However,  the  percentages  also  reflect  changes  in  the  command  style  of  the  CO 
and  XO.  On  Day  4  and  Day  5,  for  example,  the  CO  was  far  more  actively  involved  in  instructing 
the  aircraft  about  where  to  fly  and  what  to  do  than  on  Day  1  and  Day  2. 

In  future  exercises,  the  number  of  messages  in  this  class  could  serve  as  a  useful  metric  of 
command  style;  they  also  could  serve  as  useful  feedback  for  training  on  information 
management  and  command  style. 

Soot  reports.  Spot  reports  account  for  only  a  small  percentage  (4.0%)  of  the  messages 
that  occurred  on  Day  1 .  However,  the  percentage  of  spot  reports  increases  significantly  (13.2%) 
on  Day  2,  %2  (1,  487)  =  24.0,  p<  .001,  and  again  (to  25.1%)  on  Day  4,  %2  (1,  487)  =  46.1,  p< 

.001.  The  decline  in  percentage  from  Day  4  to  Day  5  is  not  large  enough  to  reach  the  .05  level  of 
confidence,  %2  (1,  487)  =  .62,  p>  .3.  The  percentages  for  Day  4  and  Day  5  are  most 
representative  of  the  contribution  of  spot  reports  to  the  volume  of  communication  events  during 
missions  that  involve  both  reconnaissance  assets  (aircraft  and  UAV)  and  attack  assets. 

It  is  important  to  note  that  spot  reports  were  sometimes  conveyed  directly  to  the  CO  and 
sometimes  to  the  BC.  Although  the  ASAS  and  MCS  operators  were  expected  to  enter  spot 
report  data  into  their  respective  systems,  there  were  few  instances  in  which  either  the  CO  or  the 
BC  overtly  conveyed  the  spot  reports  to  the  digital  systems  operators.  Because  the  messages 
were  heard  on  speakers  rather  than  headphones,  it  can  be  assumed  that  the  operators  were  able  to 
hear  most  of  the  spot  reports.  However,  there  is  no  way  the  observers  could  determine  the 
number  of  instances  in  which  the  digital  system  operators  heard  spot  reports  announced  on  the 
speakers. 

The  inability  to  track  the  flow  of  spot  report  information  to  the  digital  system  operators  is 
an  important  methodological  shortfall.  The  use  of  speakers  eliminates  the  requirement  to  convey 
spot  report  information  in  a  manner  that  would  be  required  if  head  phones  and  independent  radio 
channels  were  required.  This  leads  to  an  underestimate  of  the  volume  of  communication  events 
that  would  occur  with  conventional  voice  communication  systems  that  are  used  in  both  ground- 
based  and  airborne  TOCs.  In  future  exercises,  this  problem  could  be  eliminated  if  all  spot  report 
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information  was  conveyed  via  radio  headphones  worn  by  both  battle  staff  members  and 
observers. 

Messages  about  mission  plan/status.  This  message  class  includes  a  very  wide  variety 
of  questions  about  the  mission  plan  or  the  status  of  the  mission.  Listed  below  are  examples  of 
questions  or  announcements  that  were  classified  into  this  message  category: 

•  take-off  times,  changes  in  time  schedule,  and  time  remaining  (on  station  or  until  an 
event  is  scheduled  to  occur); 

•  location  of  important  geographic  locations  (e.g.,  FLOT,  NAIs,  FARPs); 

•  graphics  or  coordinates  of  graphic  objects; 

•  artillery  fires  planned,  ongoing,  or  completed; 

•  CO’ s  intent  (e.g.,  intent  to  bypass  troops,  intent  to  engage  target); 

•  capability  to  attack  target  or  to  direct  artillery  fire; 

•  mission  status  (e.g.,  aircraft  are  enroute,  aircraft  are  engaging  targets,  units  will  be 
moving  shortly);  and 

•  availability  of  resources  (e.g.,  UAV,  artillery,  ammunition). 

The  percentage  of  messages  about  mission  plan  and  status  is  significantly  higher  for  Day 
1  (22.2%)  than  for  Day  2,  %2  (1,  487)  -  16.8,  p<  .001,  or  Day  4,  y?  (1,  487)  =  21.8,  p<  .001,  or 
Day  5,  %2  (1,  487)  =  16.8 ,p<  .001.  These  differences  between  Day  1  and  the  following  days 
stem  mainly  from  the  many  questions  and  announcements  that  were  needed  on  Day  1  to  resolve 
battle  staff  members’  uncertainties  about  (a)  the  specific  missions  that  were  being  planned  and 
conducted  and  (b)  the  procedures  to  be  used  during  the  execution  of  missions.  The  percentages 
for  Day  2,  Day  4,  and  Day  5  do  not  differ  significantly,  y2  (1,  487)  =  .33  to  1.4,  p>  .50. 

Messages  about  aircraft  location.  Messages  in  this  category  include  both  requested  and 
unsolicited  reports  by  pilots  or  the  UAV  operators  about  the  location  of  their  aircraft.  Most 
requests  for  information  about  aircraft  location  were  made  by  the  CO  or  by  the  XO.  The 
percentages  of  messages  about  aircraft  location  do  not  differ  for  Day  1  and  Day  2,  y2  (1,  487)  = 
.87 ,p>  .50,  and  do  not  differ  for  Day  4  and  Day  5,  y2  (1,  487)  =  .10,  p>  .70.  However,  the 
percentages  are  significantly  higher  for  Day  4  (15.9%)  and  Day  5  (15.2%)  than  for  Day  1  (8.3%) 
and  Day  2  (10.1%),  x2  (1,487)  =  5.1  to  11.8,  /><  .05.  Differences  in  the  types  of  missions  flown 
on  different  days  certainly  contributed  to  differences  in  the  percentages  of  messages  about 
aircraft  location.  However,  differences  in  command  strategy  or  style  also  contributed  to  the  day- 
to-day  differences.  Increases  in  the  percentage  of  messages  that  were  orders  or  requests  to  take 
action  (see  above  discussion)  indicate  that  the  CO  and  XO  were  more  actively  involved  in 
controlling  the  actions  of  the  aircraft  on  Day  4  and  Day  5  than  on  Day  1  and  Day  2.  It  is 
reasonable  to  assume  that  more  active  control  requires  more  frequent  updates  about  the  current 
location  of  the  aircraft. 

The  percentage  of  messages  that  conveyed  information  about  aircraft  location  probably 
would  be  considerably  smaller  if  the  AMDW/S  and/or  the  FAADEO  had  been  operational  and 
capable  of  tracking  the  location  of  friendly  aircraft.  In  future  studies,  it  will  be  of  interest  to 
determine  the  extent  to  which  the  AMDW/S,  FAADEO,  or  other  digital  systems  reduce  the  . 
percentage  of  messages  that  convey  information  about  the  current  location  of  the  aircraft. 
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Questions  about  digital  systems.  Messages  in  this  category  are  either  questions  or 
answers  about  the  capability  or  operation  of  the  digital  system.  No  attempt  was  made  to  record 
the  messages  between  the  military  equipment  operator  (ASAS  or  MCS)  and  the  civilian  who 
assisted  in  operating  the  equipment.  About  71%  of  the  messages  in  this  class  were  questions 
posed  by  the  CO  or  XO.  About  21%  of  the  questions  were  posed  by  the  BC,  and  the  remaining 
7.8%  were  posed  by  another  battle  staff  member.  Equal  numbers  of  questions  were  directed  at 
the  MCS  and  ASAS.  Questions  about  the  ASAS  or  the  MCS  accounted  for  nearly  90%  of  all 
messages  in  this  class.  One  question  was  asked  about  the  UAV  display,  and  three  questions  were 
asked  about  the  JSTARS  system. 

As  would  be  expected,  the  percentages  of  messages  about  the  capabilities  of  the  digital 
systems  decline  significantly  from  Day  1  (7.7%)  to  Day  2  (4.9%),  %2  (1,  487)  =  17.6,  p<  .001, 
and  decline  significantly  from  Day  2  to  Day  4  (1.8%),  %2  (1,  487)  =  6.7,  p<  .01.  The  percentages 
for  Day  4  and  Day  5  do  not  differ  significantly,  %2  (1,  487)  =  .22,  p>  .50.  Because  most  of  the 
questions  were  posed  by  the  CO  and  XO,  this  trend  is  probably  attributable  to  skill  acquisition  by 
the  CO  and  the  XO. 

The  questions  posed  by  members  of  the  battle  staff  provide  useful  information  about 
specific  deficiencies  in  battle  staff  members’  knowledge  about  digital  systems.  However,  the 
number  of  such  questions  asked  is  not  a  valid  metric  of  the  battle  staff  member’s  individual  or 
collective  knowledge.  A  battle  staff  member’s  knowledge  about  a  digital  system  may  be  so 
meager  that  he  or  she  is  incapable  of  asking  meaningful  questions  about  its  capability  or 
function.  Furthermore,  a  battle  staff  member  may  be  reluctant  to  ask  a  question  that  reveals  his 
or  her  lack  of  knowledge. 

Requests  to  change  a  display.  Most  of  the  messages  in  this  class  are  requests  by  the  CO 
or  XO  to  (a)  change  the  system  that  appeared  on  the  large  video-wall  display  (to/from  ASAS 
to/from  MCS),  (b)  scroll  the  display  to  a  different  location,  or  (c)  zoom  the  display  in  or  out  on  a 
specific  feature  or  area.  The  data  in  Table  1 1  show  no  trends  suggesting  that  the  CO  and  XO 
learned  to  take  advantage  of  their  capability  to  change,  scroll,  and  zoom  the  ASAS  and  MCS 
displays.  Conversely,  the  data  provide  no  evidence  that  the  ASAS  and  MCS  operators  learned  to 
change,  scroll,  or  zoom  the  displays  in  anticipation  of  the  needs  of  the  CO  and  XO.  Although 
the  percentage  of  messages  in  this  class  was  significantly  less  on  Day  4  than  for  all  other  days,  x 
(1,  487)  =  3.8  to  4.8,  p<  .05,  the  reasons  for  the  decline  on  Day  4  are  not  known. 

The  messages  in  this  class  suggest  the  possible  need  to  provide  the  CO  and  XO  with 
controls  that  would  enable  them  to  directly  change,  scroll,  and  zoom.  The  number  of  messages 
in  this  class  may  serve  as  a  sensitive  measure  of  the  effectiveness  of  such  devices. 

Information  Flow/Management 

This  section  describes  the  results  of  analyses  of  the  flow  of  information  to  and  from  the 
CO  and  the  flow  of  information  to  and  from  the  XO.  The  description  of  the  results  of  these 
analyses  is  preceded  by  introductory  comments  about  the  rationale  underlying  the  analyses  that 
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were  conducted.  These  comments  should  enable  the  reader  to  better  understand  the  implications 
of  the  data  that  are  presented. 

Introductory  comments.  The  ultimate  purpose  of  the  equipment,  personnel,  and 
procedures  employed  in  a  command  and  control  center  is  to  provide  the  information  that  the  CO 
must  have  to  establish  and  maintain  an  adequate  level  of  situation  awareness.  By  definition,  an 
adequate  level  of  situation  awareness  requires  that  the  CO  have  information  of  the  type,  quantity, 
and  currency  to  perform  the  broad  functions  listed  below  (Endsley,  1995a;  Endsley,  1995b; 
Endsley,  1997;  Garland  and  Endsley,  1995;  Gilson,  Garland,  andKoonce,  1994). 

•  Perceive  all  available  information  about  battlefield  elements,  including  both 
friendly  and  opposing  forces. 

•  Comprehend  the  meaning  of  all  information  about  battlefield  elements  when 
evaluated  in  the  context  of  existing  topography,  mission,  and  command  level. 

•  Project  battlefield  elements  into  the  near-term  future  and  correctly  interpret  the 
forecast. 

Research  to  assess  the  design  and  use  of  a  command  and  control  system,  such  as  the 
ATCCS,  must  examine  the  manner  in  which  the  CO  acquires  the  information  needed  to  establish 
and  maintain  situation  awareness.  According  to  contemporary  information  theory,  there  are  two 
basic  methods  by  which  a  CO  acquires  information:  the  supply-push  method  and  the  demand- 
pull  method  (Kahan  and  Worley,  1989;  Schmitt  et  al.,  1994).  Supply-push  means  that 
information  is  conveyed  to  the  CO  (unsolicited)  as  it  becomes  available  or  in  accordance  with  a 
predetermined  procedure  or  schedule.  By  contrast,  demand-pull  means  that  information  is 
conveyed  to  the  CO  only  when  the  CO  requests  it.  Both  methods  have  advantages  and 
disadvantages.  The  main  advantages  of  the  supply-push  method  are  (a)  the  CO  does  not  need  to 
request  the  information,  and  (b)  the  information  can  be  provided  (pushed)  to  the  CO  a  short  time 
after  it  becomes  available.  The  main  disadvantage  of  the  supply-push  method  is  the  danger  that 
the  CO  will  be  inundated  with  more  information  than  any  human  can  process.  Given  the 
capability  of  contemporary  battlefield  information  systems,  information  overload  by  the  CO  is  a 
near  certainty  if  methods  are  not  developed  to  screen  the  information  flowing  into  the  command 
and  control  facility. 

The  obvious  advantage  of  the  demand-pull  method  is  that  it  minimizes  the  chances  that 
the  CO  will  be  inundated  with  more  information  than  is  needed  or  more  information  than  can  be 
processed  in  the  time  available.  The  disadvantages  of  the  demand-pull  method  are  (a)  the  CO’s 
workload  is  increased  by  the  requirement  to  request  information  and  (b)  the  system  may  contain 
important  information  that  is  not  requested  by  the  CO.  Important  information  may  not  be 
requested  because  the  CO  may  be  unaware  of  its  presence  or  because  the  CO  misjudges  its 
importance. 

In  practice,  both  the  supply-push  and  demand-pull  methods  are  necessary  and  are  used  in 
varying  degree  in  command  and  control  settings.  However,  excessive  use  or  ineffective  use  of 
either  method  is  likely  to  be  problematic.  A  clear  understanding  of  the  methods  by  which  the 
CO  acquires  information  is  a  necessary  first  step  in  understanding  the  information  management 
procedures  that  are  employed  in  a  command  and  control  center  and  the  information  management 
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problems  that  occur.  The  purpose  of  the  analyses  presented  below  was  to  gain  a  better 
understanding  of  the  information  management  methods  employed  by  the  battle  staff. 

Information  acquisition  bv  the  CO  and  XO.  The  communication  events  in  which  the 
CO  was  the  recipient  were  classified  with  respect  to  (a)  the  source  of  the  information,  (b)  the 
method  by  which  the  information  was  acquired  (push  or  pull),  and  (c)  the  content  of  the  message 
(information  or  a  recommended  action).  The  communication  events  were  then  tabulated  to 
determine,  for  each  of  eight  sources,  the  number  of  communication  events  in  which: 

•  information  was  “pulled”  (requested)  by  the  CO, 

•  information  was  “pushed”  (unsolicited)  to  the  CO,  and 

•  an  order  from  brigade  or  request  for  action  from  brigade  was  “pushed”  to  the  CO. 

It  is  important  to  remind  the  reader  that  this  analysis  of  information  acquisition  includes 
only  the  information  that  the  CO  acquired  through  voice  messages.  There  is  no  doubt  that  the 
CO  acquired  some  information  from  the  displays  that  appeared  on  the  video  wall,  but  no  data  are 
available  on  the  type  or  amount  of  information  he  acquired  from  displays.  However,  because  the 
displayed  information  was  so  difficult  to  discriminate  from  the  CO’s  position,  it  is  safe  to 
conclude  that  he  acquired  only  a  small  amount  of  information  from  the  video  wall  displays.  This 
conclusion  is  supported  by  the  fact  that  the  CO  rarely  left  his  desk  to  view  either  the  wall- 
mounted  displays  or  the  desk-mounted  displays. 

Figure  16  shows  the  number  of  communication  events  in  which  information  was  acquired 
from  each  of  the  eight  sources.  The  numbers  shown  are  a  consolidation  of  the  communication 
events  that  occurred  during  the  4  days  in  which  missions  were  conducted.  Also  shown  in  Figure 
16  are  the  relative  frequencies  with  which  information  was  pushed  from  and  pulled  from  each  of 
the  eight  sources.  Clearly,  the  most  striking  finding  is  the  very  large  amount  of  information  that 
the  CO  acquired  from  the  aircraft  pilots.  Of  the  322  communication  events  in  which  the  CO  was 
the  recipient,  166  (51.6%)  were  events  in  which  information  or  recommendations  came  from  the 
aircraft  pilots.  The  information  was  pulled  (requested  by  CO)  from  the  pilots  in  62  (37.3%) 
events  and  pushed  from  the  pilots  in  102  (71.4%)  events.  As  would  be  expected,  an  aircraft  pilot 
infrequently  (6  events,  1.2%  of  total)  pushed  a  recommended  action  to  the  CO. 
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Number  of  Communication  Events  (to  CO) 

Figure  16.  Distribution  of  communication  events  in  which  the  CO  was  the  recipient  of 
information  or  recommended  action. 

A  second  important  finding  is  the  relatively  small  amount  of  information  that  the  CO 
acquired  from  the  MCS  cell,  the  AS  AS  cell,  the  BC,  and  the  XO.  During  the  four  days  in  which 
missions  were  conducted,  there  were  only  18  (5.6%)  communication  events  in  which  the  CO 
acquired  information  from  the  MCS  cell  and  only  11  (3.4%)  communication  events  in  which  he 
acquired  information  from  the  AS  AS  cell.  It  might  be  assumed  that  the  BC  and/or  XO  served  as 
an  intermediary  in  conveying  information  from  the  digital  system  operators  (MCS  and  ASAS)  to 
the  CO.  This  assumption  is  not  supported  by  the  data.  There  were  only  21  (6.5%) 
communication  events  in  which  the  CO  acquired  information  from  the  BC  and  only  20  (6.2%) 
communication  events  in  which  he  acquired  information  from  the  XO.  Except  for  that  ASAS 
cell,  information  was  pushed  to  the  CO  more  often  than  it  was  pulled.  It  is  clear  from  these 
findings  that  the  CO  acquired  (directly  or  indirectly)  a  relatively  small  amount  of  information 
from  the  digital  systems. 

Communication  events  in  which  the  XO  was  the  recipient  were  classified  and  tabulated 
in  the  same  manner  as  that  described  above  for  the  CO.  The  results  are  shown  in  Figure  17.  The 
same  ordinate  scale  was  used  in  Figure  16  and  Figure  17  to  facilitate  comparison  of  the  number 
and  distribution  of  communication  events  in  which  the  CO  and  XO  were  the  recipients. 
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The  XO  was  the  recipient  of  far  fewer  communication  events  (150)  than  the  CO  (322),  %2 
(1, 472)  =  62.7,  p<  .001.  However,  the  distribution  of  the  XO’s  information  sources  is  similar  to 
the  distribution  of  the  CO’s  information  sources.  In  particular,  aircraft  pilots  were  the  source  of 
48%  of  the  communication  events  in  which  the  XO  was  the  recipient  of  information.  In  addition, 
the  XO  acquired  a  relatively  small  amount  of  information  from  the  MCS  station  (7 
communication  events),  the  AS  AS  station  (18  communication  events),  or  from  the  BC  (13 
communication  events).  However,  the  CO  and  XO  differed  in  the  method  by  which  their 
information  was  acquired.  Information  pull  accounts  for  66.8%  of  the  communication  events  in 
which  the  XO  was  the  recipient  but  only  3 1 .4%  of  the  information  events  in  which  the  CO  was 
the  recipient.  In  short,  the  XO  acquired  most  of  his  information  by  requesting  it  (information 
pull)  while  the  CO  acquired  most  of  his  information  through  unsolicited  messages  from  various 
sources  (information  push). 


Number  of  Communication  Events  (to  XO) 


Figure  17.  Distribution  of  communication  events  in  which  the  XO  was  the  recipient  of 
information  or  recommended  action. 


Information  dissemination  by  the  CO  and  XO.  The  communication  events  originated 
by  the  CO  were  classified  in  terms  of  the  recipient  of  the  communication.  The  communication 
events  for  each  recipient  were  further  classified  into  three  categories:  information  pull, 
information  push,  and  action  push.  Information  pull  means  that  the  CO  conveyed  information  in 
response  to  the  recipient’s  request  for  information.  Information  push  means  that  the  CO 
conveyed  information  that  was  not  solicited  by  the  recipient.  Action  push  means  that  the  CO 
conveyed  an  order  or  a  recommended  action  to  the  recipient. 
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Figure  18  shows  the  distribution  of  communication  events  in  which  the  CO  disseminated 
information.  Aircraft  pilots  were  the  recipients  of  142  (56.1%)  of  the  253  communication  events 
for  which  the  CO  disseminated  information  or  an  order.  The  CO  infrequently  (4  communication 
events)  disseminated  information  to  a  pilot  in  response  to  the  pilot’s  request  for  information 
(information  pull).  Similarly,  unsolicited  dissemination  of  information  by  the  CO  (information 
push)  accounts  for  only  26.8%  of  the  communication  events  in  which  a  pilot  was  the  recipient. 

In  contrast,  the  dissemination  of  orders  by  the  CO  (action  push)  accounts  for  70.4%  of  the 
communication  events  in  which  a  pilot  was  the  recipient.  These  data  provide  further  evidence  of 
the  CO’s  active  involvement  in  directing  the  action  of  the  aircraft  during  the  conduct  of  the 
missions. 


Number  of  Communication  Events  (from  CO) 

Figure  18.  Distribution  of  communication  events  in  which  the  CO  disseminated  information  to 
another  battle  staff  member  or  to  another  battlefield  element. 

The  data  in  Figure  18  provide  no  evidence  that  the  CO  routinely  disseminated 
information  through  intermediary  staff  members.  There  were  instances  in  which  the  CO  asked 
the  XO  or  the  BC  to  communicate  information  to  brigade,  aircraft  pilots,  and  the  UAV  operator. 
However,  this  was  not  a  routine  method  for  disseminating  information.  The  XO  was  the 
recipient  of  only  3.6%  of  the  communication  events  for  which  the  CO  disseminated  information; 
the  BC  was  the  recipient  of  only  5.9%  of  the  communication  events  in  which  the  CO 
disseminated  information. 

During  the  4  days  in  which  missions  were  conducted,  there  were  only  107 
communication  events  in  which  the  XO  was  the  originator.  These  communication  events  were 
classified  in  the  same  manner  as  for  the  communication  events  in  which  the  CO  was  the 
originator.  The  results  are  shown  in  Figure  19.  To  facilitate  comparison,  the  same  ordinate  scale 
was  used  for  Figure  1 8  and  Figure  19.  As  was  true  for  the  CO,  most  of  the  information 
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disseminated  by  the  XO  was  unsolicited  by  the  recipient.  The  dissemination  of  unsolicited 
information  by  the  XO  (information  push)  accounts  for  53.3%  of  the  communication  events; 
disseminating  information  in  response  to  a  request  (information  pull)  accounts  for  only  15.9%  of 
the  communication  events.  The  most  frequent  recipients  of  information  from  the  XO  were  the 
CO  (33.6%),  the  manned  aircraft  (18.7%),  the  ASAS  cell  (16.8%),  and  the  BC  (15.9%).  No 
other  recipient  accounts  for  more  than  7.5%  of  the  communication  events. 


Number  of  Communication  Events  (from  XO) 

Figure  19.  Distribution  of  communication  events  in  which  the  XO  disseminated  information  to 
another  battle  staff  member  or  to  another  battlefield  element. 

Information  received  bv  and  relayed  bv  the  BC.  A  potential  role  of  a  BC  is  to  reduce 
the  CO’s  voice  communication  workload  by  receiving  messages  from  battlefield  elements  and 
relaying  relevant  information  to  the  CO.  The  data  were  tabulated  to  determine  the  number  of 
communication  events  in  which  the  BC  was  the  recipient  of  information  that  was  subsequently 
relayed  to  either  the  CO  or  XO.  There  were  58  communication  events  in  which  the  BC  was  the 
recipient  of  information  sent  from  brigade.  In  only  8  events  (13.7%)  did  the  BC  convey  the 
information  received  from  brigade  to  the  CO  or  to  the  XO.  Of  the  104  communication  events  in 
which  the  BC  was  the  recipient  of  information  sent  from  an  aircraft,  there  was  only  one 
communication  event  in  which  the  BC  relayed  the  information  to  the  CO  or  to  the  XO. 

Much  of  the  information  received  from  brigade  and  from  the  aircraft  was  sufficiently 
important  to  warrant  relay  to  the  CO  or  XO,  so  the  failure  to  do  so  does  not  reflect  prudent 
screening  by  the  BC.  It  is  possible  that  the  BC  assumed  that  the  CO  or  XO  heard  the  messages 
because  they  were  transmitted  to  a  speaker  located  on  the  BC’s  desk.  Regardless  of  the  reasons, 
however,  these  data  indicate  that  suitable  TTPs  were  not  established  to  guide  the  BC  in 
managing  the  information  conveyed  to  him  from  other  battlefield  elements. 
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Post-Exercise  Training  Effectiveness  Survey 

This  subsection  describes  battle  staff  members’  responses  to  the  Training  Effectiveness 
Questionnaire  (see  Appendix  E)  that  every  battle  staff  member  completed  following  the  last 
mission  execution  exercise.  The  questionnaire  has  5  items  for  each  of  six  topical  areas  (scales) 
entitled  learning,  exercise  conduct,  future  application,  digital  system  utility,  instructional 
feedback,  and  enjoyment.  The  items  developed  for  each  scale  are  shown  on  the  last  page  of 
Appendix  E.  Even  at  first  glance,  however,  it  was  clear  that  the  researchers'  construction  of 
dimensions  underlying  the  items  on  this  survey  instrument  did  not  provide  a  representative 
model  of  the  dimensions  along  which  the  participants  responded.  Items  that  should  have 
exhibited  correlation  as  a  result  of  membership  in  a  common  scale  showed  essentially  unrelated 
results.  In  short,  the  underlying  structure  of  the  survey  proved  to  be  invalid.  There  were, 
however,  consistent  patterns  of  responding  evident  in  the  data. 

Because  of  the  manner  in  which  battle  staff  members  responded  to  the  items,  the  items 
have  been  grouped  in  a  different  way  in  the  following  description  of  the  findings.  In  the 
following  subsection,  the  items  were  grouped  into  four  sets.  The  first  set  includes  items  that 
reflect  battle  staff  members’  opinions  about  the  training  benefits  of  the  ATCCS  exercise.  The 
second  and  third  sets  include  items  that  reflect  battle  staff  members’  opinions  about  the  conduct 
of  the  exercises  and  the  utility  of  the  digital  systems,  respectively.  The  fourth  set  includes  items 
that  reflect  the  degree  to  which  battle  staff  members  enjoyed  participating  in  the  exercises.  This 
organization  of  items  better  fit  the  dimensions  along  which  participants  appear  to  have 
interpreted  the  items. 

Training  Benefits 

The  data  in  Table  8  show  battle  staff  members’  agreement  with  12  statements  about  the 
training  benefits  of  the  ATCCS  exercise.  The  numbers  in  the  cells  adjacent  to  each  statement  are 
the  percentages  of  battle  staff  members  who  selected  the  corresponding  rating-scale  alternative. 
These  data  leave  no  doubt  that,  as  a  group,  battle  staff  members  believed  that  they  realized 
training  benefits  from  participating  in  the  ATCCS  exercise.  There  is  a  very  high  level  of 
agreement  with  each  of  the  first  1 1  statements  listed  in  Table  8.  For  each  of  these  statements,  no 
fewer  than  64%  of  the  battle  staff  members  agreed  (mildly  or  strongly)  and  no  more  than  18% 
disagreed  (strongly  or  mildly)  with  the  statement.  Because  the  training  was  not  aimed 
specifically  at  leadership  skills,  it  is  not  surprising  that  only  36%  of  the  battle  staff  members 
agreed  mildly  or  strongly  with  the  statement  “This  training  helped  my  leadership  skills.” 
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Table  8 


Battle  staff  members’  opinions  about  the  training  benefits  of  the  ATCCS  exercise. 


Rating-Scale  Alternatives 

Survey  Statement 

Strongly 

Disagree 

Mildly 

Disagree 

Neutral 

Mildly 

Agree 

Strongly 

Agree 

I  know  more  now  than  I  did  before  the  exercise 

- 

-- 

- 

45% 

55% 

This  was  a  good  learning  experience  for  me 

- 

9% 

-- 

82% 

9% 

I  acquired  new  skills  from  the  ATCCS  exercise 

- 

9% 

- 

91% 

- 

The  ATCCS  exercise  was  a  useful  learning  experience 

- 

9% 

- 

82% 

9% 

I  feel  I  gained  substantial  benefit  from  the  ATTCS  exercise 

9% 

- 

9% 

55% 

27% 

I  intend  to  use  what  I  learned  during  the  exercise  in  future  assignments 

- 

9% 

9% 

64% 

18% 

I  believe  that  this  experience  will  help  me  perform  better  as  a  staff  officer 

-- 

9% 

18% 

45% 

27% 

I  have  a  better  level  of  ability  as  a  staff  officer  because  of  this  exercise 

- 

9% 

27% 

64% 

- 

I  learned  a  great  deal  during  the  ATCCS  exercise 

- 

9% 

27% 

55% 

9% 

I  intend  to  apply  what  I  learned  in  field  assignments 

- 

- 

36% 

64% 

- 

This  training  will  have  a  positive  effect  on  my  future  performance 

9% 

9% 

18% 

64% 

- 

This  training  helped  improve  my  leadership  skills 

9% 

18% 

36% 

27% 

9% 

Conduct  of  Exercise 

Table  9  shows  battle  staff  members’  agreement  with  each  of  nine  statements  about  the 
manner  in  which  the  exercises  were  conducted.  Between  63%  and  91%  of  the  responses 
indicated  that  they  agreed  mildly  or  strongly  with  the  first  three  statements  listed  in  Table  9.  So, 
most  battle  staff  members  shared  the  opinion  that  (a)  use  of  the  ATCCS  suite  promoted  practice 
of  relevant  skills,  (b)  the  AAR  process  was  a  useful  learning  tool,  and  (c)  the  feedback  received 
during  the  exercises  was  important.  However,  no  more  than  45%  and  as  few  as  9%  of  the  battle 
staff  members  agreed  mildly  or  strongly  with  any  of  the  remaining  statements  listed  in  Table  9. 
There  was  no  consistency  in  battle  staff  members’  agreement  with  the  statements  that  (a)  the 
observers  provided  beneficial  instruction,  (b)  the  scenarios  simulated  battlefield  conditions,  (c) 
the  ATCCS  exercise  lasted  the  right  amount  of  time,  or  (d)  the  observers  provided  good  analyses 
of  the  battle  staff  performance.  For  all  four  items,  the  percentage  who  agreed  is  about  the  same 
as  the  percentage  who  disagreed. 

There  is  more  consistency  in  battle  staff  members’  responses  to  the  last  two  statements 
listed  in  Table  9.  About  55%  of  the  battle  staff  members  disagreed  mildly  with  the  statement 
“The  ATCCS  environment  was  well  designed.”  Battle  staff  members’  opinions  about  the 
ATCCS  design  probably  would  have  been  more  favorable  if  all  the  digital  systems  had  been 
functioning  properly  during  all  of  the  exercises.  Inadequate  training  on  the  function  and 
operation  of  the  digital  systems  is  the  most  likely  reason  why  72%  of  the  battle  staff  members 
disagreed  strongly  or  mildly  with  the  statement  “I  was  well  prepared  for  this  training  exercise.” 
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Table  9 

Battle  staff  members’  opinions  about  the  conduct  of  the  ATCCS  exercise. 


Rating-Scale  Alternatives 

Survey  Statement 

Strongly 

Disagree 

Mildly 

Disagree 

Neutral 

Mildly 

Agree 

Strongly 

Agree 

Skills  required  in  the  ATCCS  exercise  are  important  during  field  exercises 

-- 

9% 

- 

55% 

39% 

The  AAR  process  was  a  useful  learning  tool 

-- 

9% 

18% 

45% 

27% 

The  feedback  I  received  during  this  exercise  was  very  important 

- 

18% 

18% 

55% 

9% 

Observers/controllers  provided  beneficial  instruction 

18% 

18% 

18% 

36% 

9% 

The  ATCCS  scenarios  well  simulated  actual  battlefield  engagements 

18% 

18% 

27% 

36% 

- 

The  ATCCS  lasted  the  right  amount  of  time  to  be  useful 

18% 

27% 

18% 

27% 

9% 

The  observers  provided  good  analyses  of  the  battalion  staff  performance 

- 

27% 

36% 

27% 

9% 

The  ATCCS  environment  was  well  designed 

- 

55% 

18% 

27% 

-- 

I  was  well  prepared  for  this  training  exercise 

27% 

45% 

18% 

- 

9% 

Dfeital  System  Utility 

Battle  staff  members  rated  their  agreement  with  five  statements  about  digital  system 
utility.  Most  battle  staff  members  agreed  mildly  or  strongly  to  the  abstract  statements  “The 
digital  TOC  enhances  battalion  staff  capabilities”  (73%  agreed)  and  “The  ATCCS  suite  provides 
significant  improvement  in  information  exchange”  (55%  agreed).  However,  the  majority  of 
battle  staff  members  disagreed  strongly  or  mildly  with  the  more  specific  statements  “Mission 
planning  is  faster  in  the  digitized  TOC  (82%  disagreed),  “The  exchange  of  information  through 
digitized  communications  was  easy"  (73%  disagreed),  and  “Planning  and  executing  missions  is 
easier  in  a  digitized  TOC”  (55%  disagreed).  The  lack  of  training  on  the  digital  systems  probably 
was  the  main  reason  why  battle  staff  members  disagreed  with  statements  that  the  digital  systems 
saved  time  and  that  the  digital  systems  made  it  easier  to  exchange  information  and  to  plan  and 
execute  missions. 

Enjoyment 

Battle  staff  members  rated  their  agreement  with  four  statements  about  the  degree  to 
which  they  enjoyed  their  participation  in  the  ATCCS  exercise.  It  was  assumed  that  battle  staff 
members  would  disagree  with  these  statements  if  they  judged  that  participation  was  excessively 
stressful,  boring,  or  tiring.  Listed  below  are  the  four  statements  in  this  set  and  the  percentage  of 
battle  staff  members  who  disagreed  or  agreed  with  each  statement. 

•  The  ATCCS  exercise  was  interesting  and  enjoyable  (18%  disagreed  and  82%  agreed). 

•  I  would  like  to  participate  in  other  similar  exercises  (18%  disagreed  and  64%  agreed). 

•  This  exercise  exceeded  my  expectations  (27%  disagreed  and  45%  agreed). 

•  I  enjoyed  myself  during  this  exercise  (18%  disagreed  and  36%  agreed). 
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The  battle  staff  members’  responses  to  these  items  provide  no  evidence  that  the  ATCCS 
exercise  was  so  unpleasant  that  they  would  be  unsuitable  for  further  research  or  for  battle  staff 
training. 


Synopsis  of  Reported  Problems 

Throughout  this  research,  battle  staff  members  were  asked  to  describe  problems  they 
observed  or  experienced  during  mission  planning  or  mission  execution.  Battle  staff  members’ 
descriptions  of  problems  come  from  three  sources.  One  source  is  battle  staff  members’ 
responses  to  the  open-end  questionnaire  items  that  invited  them  to  express  their  views  about  a 
variety  of  topics  and  issues.  A  second  source  is  the  observers’  notes.  During  both  mission 
planning  and  mission  execution,  observers  recorded  battle  staff  members’  comments  about  the 
problems  they  observed  or  experienced.  The  third  source  is  interviews  with  the  XO,  SI,  S3  and 
S4,  which  were  conducted  the  week  after  the  study  had  been  completed  (March  9-13,  1998).  A 
large  number  and  wide  variety  of  problems  are  revealed  in  these  three  sources.  In  addition,  the 
problems  were  expressed  in  a  variety  of  ways.  In  some  instances,  battle  staff  members  described 
solutions  rather  than  problems.  In  such  instances  it  was  necessary  to  infer  problems  from  the 
solution  that  was  offered.  In  other  instances,  battle  staff  members  described  problems  that  were, 
in  fact,  symptoms  of  a  more  fundamental  problem. 

The  problem  descriptions  presented  in  this  subsection  are  a  synopsis  of  the  problems 
identified  by  the  battle  staff  members.  The  problems  are  subdivided  into  two  groups.  The  first 
group  includes  problems  that  bear  on  the  research  methods  and  facilities  that  were  used.  The 
second  group  includes  a  variety  of  problems  that  bear  on  the  design  of  the  ATCCS  equipment. 
All  of  the  equipment  design  problems  can  be  related  to  problems  that  battle  staff  members 
encountered  in  performing  tasks  during  mission  planning  or  mission  execution.  Although 
improved  training  or  improved  procedures  may  be  a  partial  solution  for  some  of  the  problems, 
battle  staff  members’  comments  indicate  clearly  that  improvements  in  the  design  of  system 
hardware  or  software  were  considered  to  be  the  preferred  solution. 

Research  Methods  and  Facilities 

Training.  A  problem  identified  by  virtually  all  battle  staff  members  is  the  lack  of 
sufficient  prior  training.  Battle  staff  members  stated  they  should  have  had  more  training  on  both 
the  capability  and  operation  of  the  digital  systems.  They  also  stated  that  more  training  was 
needed  to  enable  them  to  work  together  effectively.  Battle  staff  members  expressed  the  view 
that  much  more  could  have  been  learned  about  the  use  and  utility  of  digital  systems  if  they  had 
received  at  least  one  more  week  of  training  exercises  in  the  ATCCS. 

Battle  staff  members  experienced  problems  in  performing  terrain  line-of-sight  (LOS) 
analyses  with  the  digital  systems  and  problems  in  interpreting  the  results  of  the  analyses.  Battle 
staff  members  indicated  that  these  problems  would  not  have  occurred  if  they  had  adequate  prior 
training  on  (a)  the  use  of  the  digital  systems  to  perform  LOS  analyses  and  (b)  the  precision  with 
which  LOS  analyses  can  be  performed  with  the  digital  systems.  Battle  staff  members  also  stated 
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that  they  should  have  had  more  training  on  the  use  of  the  digital  systems  to  evaluate  ingress 
routes  and  egress  routes  quickly  and  effectively  during  mission  execution. 

Battle  staff  members  acknowledged  the  value  of  having  civilian  operators  available  to 
train  the  military  AS  AS  operators  and  MCS  operators.  However,  the  observers  noted  that  the 
civilian  operators  encountered  difficulty  performing  some  tasks  because  of  their  lack  of  recent 
practice  in  operating  the  system. 

Procedures.  The  root  cause  of  several  of  the  problems  described  by  battle  staff  members 
was  a  lack  of  operating  procedures  or  SOPs.  One  set  of  problems  stems  from  inadequate 
procedures  for  determining  the  information  that  should  be  displayed  on  the  large  format  wall- 
mounted  display  and,  to  a  lessor  extent,  the  desk-mounted  displays  (ASAS  and  MCS).  Battle 
staff  members  stated  that  procedures  must  be  developed  to  avoid  the  following  problems. 

•  The  CO  and  XO  were  unable  to  see  the  map  display  and  graphics  because  it  was 
obscured  by  a  pull-down  menu  or  by  another  window  (e.g.,  message  window). 

•  The  CO  and  XO  often  found  it  necessary  to  request  operators  to  change  the  system 
that  was  displayed  on  the  main  wall-mounted  display. 

•  The  CO  and  XO  often  found  it  necessary  to  direct  the  digital  system  operators  to 
scroll  a  map  display  to  a  different  location  and  to  zoom  in  or  out  on  a  specific 
geographical  location  of  a  feature.  The  CO,  XO,  and  digital  system  operators 
exhibited  considerable  uncertainty  about  the  best  display  magnification  (zoom  level) 
to  accomplish  different  tasks  or  different  mission  phases. 

A  second  set  of  problems  stems  from  the  lack  of  effective  information  management 
procedures.  In  particular,  no  clear  procedures  were  established  for  processing  either  in-coming 
or  out-going  information.  As  stated  earlier,  the  BC  was  the  recipient  of  a  substantial  amount  of 
information  (e.g.,  spot  reports  from  aircrews  and  the  UAV  operator)  that  was  not  overtly  relayed 
to  the  CO  or  XO.  In  addition,  no  consistent  procedures  were  followed  in  conveying  information 
to  external  elements  or  in  receiving  information  from  external  elements.  At  different  times, 
information  was  conveyed  to  external  elements  and  received  from  external  elements  (brigade, 
the  aircrews,  and  the  UAV  operator)  by  the  CO,  the  XO,  the  BC,  and  the  S3.  Finally,  no 
procedures  were  established  that  specified  the  responsibilities  for  monitoring  the  digital  system 
displays  or  the  UAV  display.  Most  importantly,  no  procedures  were  established  that  specified 
the  responsibilities  of  the  digital  system  operators  (ASAS  and  MCS)  for  monitoring  their 
displays,  identifying  information  needed  by  the  CO  (screening  information),  and  conveying 
(pushing)  the  information  to  the  CO. 

A  third  set  of  problems  stems  from  the  lack  of  an  effective  graphics  management 
procedure.  Because  of  problems  they  experienced,  battle  staff  members  stated  that  a  procedure 
is  needed  for  (a)  identifying  at  which  command  level  the  graphics  were  created  (brigade, 
battalion,  or  company),  (b)  deconflicting  graphics  created  at  different  command  levels,  and  (c) 
processing  changes  in  graphics.  Battle  staff  members  acknowledged  that  it  may  not  be  possible 
to  develop  effective  graphics  management  procedures  without  changes  in  the  digital-system 
hardware,  software,  or  both. 
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Finally,  problems  experienced  by  battle  staff  members  led  them  to  recommend  the 
establishment  of  the  specific  procedures  listed  below: 

•  procedures  to  ensure  that  all  information  from  prior  missions  has  been  removed  from 
the  ASAS  and  MCS, 

•  procedures  to  ensure  that  all  echelons  employ  a  common  file-naming  method  and  a 
common  date-time  labeling  method,  and 

•  procedures  to  ensure  that  all  files  transferred  among  systems  or  echelons  are 
accompanied  by  a  free  text  message  that  describes  the  contents  of  the  file. 

Planning  time.  Battle  staff  members  stated  that  the  amount  of  time  available  between 
the  receipt  of  an  OPORD  or  FRAGO  and  the  commencement  of  mission  execution  was 
inadequate  for  thorough  mission  planning.  The  S3  believed  strongly  that  more  planning  time 
should  have  been  made  available  until  members  of  the  battle  staff  learned  more  about  the  use  of 
the  digital  systems  for  performing  planning  tasks  and  until  members  of  the  battle  staff  became 
accustomed  to  working  together.  In  one  instance,  the  CO  stated  that  an  unrealistically  small 
amount  of  time  was  made  available  between  the  receipt  of  a  FRAGO  and  the  time  at  which  he 
was  expected  to  launch  his  aircraft.  This  is  a  subject  area  in  which  operational  requirements  and 
instructional  technique  conflict.  Current  Army  policy  appears  to  direct  substantial  reduction  in 
the  timeline  of  the  mission  planning  cycle  from  division  through  company  level.  An  implied 
assumption  of  this  policy  is  a  significant  increase  in  the  rate  at  which  mission  planning  functions 
can  be  completed  as  a  result  of  computer  based  tool  sets  like  ATCCS.  The  tools,  however,  are 
unlikely  to  provide  enhancements  to  planning  unless  the  users  are  proficient  in  their 
employment.  Proficiency  implies  training  which  requires  time.  While  prospective  battle  staff 
personnel  are  engaged  in  the  process  of  acquiring  this  proficiency  it  may  be  necessary  to  plan  for 
greatly  extended  mission  planning  cycles  and  the  opportunity  for  repetition  of  some  tasks  in 
order  to  facilitate  skill  acquisition,  performance  assessment,  diagnosis  and  performance 
feedback. 

Facilities  and  equipment.  Battle  staff  members  identified  three  shortfalls  in  the  ATCCS 
facility  and  equipment  that  they  judged  to  be  especially  important.  First,  the  space  provided  for 
mission  planning  was  judged  to  be  far  too  small  to  accommodate  the  mission  planning 
equipment  and  personnel.  This  problem  was  solved  on  the  first  day  of  the  study  by  moving  the 
planning  cell  to  a  larger  room.  Second,  the  battle  staff  members  expressed  the  view  that  the 
voice  communication  equipment  was  so  unrealistic  that  it  fostered  maladaptive  communication 
procedures.  Battle  staff  members  stated  that  (a)  the  voice  communication  equipment  provided 
was  not  an  acceptable  simulation  of  the  radio  systems  they  use,  and  (b)  some  battle  staff 
members  had  no  voice  communication  equipment  of  their  own.  Finally,  battle  staff  members 
stated  that  important  equipment  was  not  available  in  the  ATCCS  facility  or  was  not  operational 
during  all  or  part  of  the  study.  Some  battle  staff  members  expressed  the  view  that  the  lack  of  an 
Aviation  Mission  Planning  System  (AMPS)  contributed  to  the  use  of  some  unrealistic  procedures 
during  both  mission  planning  and  mission  execution.  Battle  staff  members  also  expressed  the 
view  that  the  procedures  they  used  during  both  mission  planning  and  mission  execution  would 
have  been  different  if  the  AMDW/S,  FAADEO,  and  AFATDS  had  been  operational  and  manned 
by  a  trained  operator. 
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Scenario  design.  Battle  staff  members  identified  attributes  of  the  scenario  design  that 
were  sufficiently  unrealistic  to  be  considered  problematic.  One  attribute  was  the  volume  of 
intelligence  information  (other  than  spot  reports)  that  was  conveyed  to  the  ATCCS  prior  to  and 
during  mission  execution.  Battle  staff  members  stated  that  the  low  volume  of  intelligence 
information  resulted  in  a  workload  level  that  was  unrealistically  low  for  some  members  of  the 
battle  staff.  A  second  attribute  that  battle  staff  members  considered  highly  unrealistic  was  that 
CSS  functions  were  not  required  or  were  too  easy  to  perform  during  both  mission  planning  and 
mission  execution.  As  a  consequence,  the  workload  level  was  unrealistically  low  for  the  SI  and 
S4. 

Eauipment/Svstem  Design 

System  displays.  Battle  staff  members  experienced  a  variety  of  problems  in  extracting 
information  from  the  system  displays.  Although  some  problems  stem  from  a  lack  of  training, 
most  problems  are  related  to  the  design  of  the  display  or  the  design  of  displayed  information. 
There  were  many  instances  in  which  battle  staff  members  were  unable  to  discriminate  the 
information  portrayed  on  the  large  wall-mounted  display.  These  problems  are  not  simply  the 
result  of  viewing  the  wall-mounted  display  from  an  excessive  distance.  The  BC,  who  sat  only  a 
few  feet  from  the  display,  complained  that  he  experienced  difficulty  reading  messages  and  could 
not  discriminate  any  map  information  from  his  position.  Although  research  will  be  required  to 
determine  how  best  to  improve  the  wall-mounted  display,  it  is  clear  that  significant 
improvements  cannot  be  achieved  by  simply  increasing  its  size  or  decreasing  viewing  distance. 
The  necessary  improvements  probably  will  require  increasing  brightness,  brightness  contrast, 
color  contrast,  and  the  number  of  addressable  lines. 

Other  problems  are  applicable  to  both  the  wall-mounted  and  desk-mounted  displays. 
Problems  identified  that  are  common  to  two  or  more  displays  are  listed  below. 

•  The  linear  symbols  (including  graphics)  depicted  on  the  displays  often  were  so  large 
that  they  obscure  too  much  map  detail.  This  problem  is  particularly  severe  when 
small-scale  maps  were  displayed. 

•  There  is  inadequate  brightness  and/or  color  contrast  to  enable  users  to  easily 
discriminate  graphics  that  overlay  map  displays. 

•  For  some  types  of  map  displays,  it  is  difficult  to  discriminate  grid  lines.  Factors  that 
may  contribute  to  this  problem  include  narrow  line  width,  low  brightness,  and  low 
brightness  contrast. 

•  Some  point  symbols  and  alphanumeric  characters  are  difficult  to  discriminate  because 
an  insufficient  number  of  lines  are  used  to  portray  them. 

Tracking  friendly  aircraft.  Both  the  CO  and  XO  acknowledged  that  it  was  very 
difficult  to  track  the  location  of  friendly  aircraft  and  to  determine  the  location  of  downed  aircraft. 
Despite  the  use  of  numerous  control  measures,  a  large  amount  of  voice  communication  was 
required  to  remain  reasonably  knowledgeable  about  the  current  location  of  the  aircraft.  The 
analysis  of  communication  events,  discussed  earlier,  supports  the  CO’s  and  XO’s  observations 
about  the  difficulty  of  tracking  friendly  aircraft. 
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Because  the  AMDW/S  was  either  not  operational  or  unmanned  during  this  study,  no 
attempt  was  made  to  use  the  AMDW/S  to  track  the  location  of  friendly  aircraft.  Although  this 
study  provided  little  information  about  the  use  and  utility  of  the  friendly  aircraft,  the  XO  made 
the  observation  that  the  capability  to  attach  identification  “tags”  to  the  friendly  aircraft  symbols 
would  increase  the  system’s  utility  for  tracking  the  location  of  friendly  aircraft. 

Maintaining  awareness  of  threat  forces.  Maintaining  an  awareness  of  the  current 
location  of  threat  vehicles  is  a  problem  that  was  experienced  by  the  CO,  the  XO,  and  probably 
other  battle  staff  members  as  well.  Three  related  problems  are  acknowledged  explicitly  or  are 
implied  by  suggestions  about  system  design  improvements.  The  overriding  problem  is  that  it  is 
very  difficult  to  process  the  ongoing  flow  of  spot  reports  and  to  integrate  this  information  with 
the  intelligence  information  available  at  the  outset  of  the  mission.  This  problem  becomes  even 
more  severe  when  aircrews  and  the  UAV  operator  commence  sending  BD  A  reports  to  the 
ATTCS. 

Although  the  CO  and  XO  acknowledged  that  improved  training  and  procedures  would 
help  remediate  this  problem,  they  indicated  that  increased  automation  is  also  required.  During  an 
interview,  the  XO  recommended  the  design  improvements  listed  below. 

•  Develop  a  capability  to  automatically  update  the  ASAS  and  MCS  when  threats  are 
observed  by  the  aircrews  or  by  the  UAV  operator,  including  spot  reports  transmitted 
by  the  Automatic  Target  Handover  System  (ATHS). 

•  Develop  a  capability  to  facilitate  the  discrimination  of  active  and  destroyed  threats. 
For  instance,  it  was  suggested  that  a  capability  be  developed  to  change  (manually  or 
automatically)  the  color  of  a  threat  symbol  when  a  report  is  received  that  the  threat 
has  been  destroyed. 

•  Develop  a  capability  to  facilitate  the  discrimination  of  (a)  multiple  reports  of  the  same 
threat  and  (b)  reports  of  multiple  threats.  It  was  recommended  that  a  computer-based 
decision  aid  be  developed  that  predicts  whether  a  threat  vehicle  could  have  traveled 
the  distance  between  two  spot  report  locations  in  the  time  that  has  transpired  between 
the  first  and  second  spot  report. 

•  Develop  a  capability  that  enables  users  to  easily  determine  the  age  of  intelligence 
information,  especially  information  about  threat  type  and  location.  It  was 
recommended  that  computer  software  be  developed  to  automatically  change  the 
appearance  of  threat  symbols  or  to  remove  them  from  the  display  when  intelligence 
information  exceeds  a  prescribed  age. 

Tailoring  type  and  format  of  displayed  information.  During  an  interview  with  the 
XO,  he  stated  that  display  clutter  is  excessive  when  all  overlay  information  is  portrayed  at  the 
same  time.  He  suggested  that  the  best  solution  to  this  problem  would  be  to  develop  a  capability 
to  select  and  display  any  one  or  more  of  a  predefined  set  of  overlay  formats.  The  XO  expressed 
the  view  that  a  group  of  experts  could  probably  define  a  small  set  of  overlay  formats  that, 
together,  would  meet  users’  needs  for  the  full  ranges  of  missions,  mission  phases,  and 
contingency  situations. 
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The  CO  stated  that  regimental  icons  do  not  meet  the  needs  of  company  commanders.  He 
expressed  the  view  that  the  systems  should  be  designed  such  that,  on  request,  the  regimental 
icons  are  deaggregated  and  symbols  are  portrayed  that  depict  the  locations  of  the  companies  that 
comprise  the  regiment. 

Interoperability  of  systems.  Battle  staff  members  described  a  host  of  problems  that  are 
the  direct  or  indirect  result  of  inadequate  interoperability  of  the  AS  AS  and  MCS.  As  was 
discussed  earlier,  problems  were  encountered  in  transferring  graphic  information  between  the 
ASAS  and  MCS.  The  lack  of  system  interoperability  contributed  to  these  problems  (along  with 
inadequate  training  and  procedures). 

The  equipment  operators  shared  the  view  that  all  digital  systems  should  employ  the  same 
topographic  map  databases  and  the  same  map  scales.  However,  they  were  not  questioned 
systematically  about  the  specific  problems  that  were  caused  by  the  lack  of  commonality  in  map 
databases  and  scales. 

It  can  be  expected  that  the  ASAS  and  MCS  will  both  be  accessed  routinely  by  some 
battle  staff  members.  It  would  be  beneficial  if  the  two  systems  employed  the  same  scheme  for 
encoding  information  that  appears  on  both  systems.  No  attempt  was  made  to  assess  the  degree 
of  commonality  in  the  two  systems’  encoding  schemes.  However,  a  few  instances  were  observed 
in  which  the  CO  or  XO  was  confused  by  differences  in  the  color-coding  schemes  employed  on 
the  two  systems. 

Map  detail.  The  need  for  improved  training  and  procedures  for  changing  the  display 
magnification  (zooming  in  and  out)  was  discussed  earlier.  However,  battle  staff  members 
suggested  that  system  design  changes  are  needed  to  provide  the  level  of  map  detail  that  is 
sometimes  needed.  When  the  map  database  remains  the  same,  zooming  in  decreases  the  viewing 
area  and  increases  the  size  of  map  features  but  does  not  increase  the  number  or  type  of  map 
features  that  are  portrayed.  At  present,  users  can  gain  access  to  more  map  detail  only  by 
changing  to  a  database  compiled  from  a  larger  scale  map.  This  condition  is  attributable  to  the 
use  of  raster  scanned  map  images  in  MCS  and  ASAS.  Battle  staff  members  recommended  that 
systems  be  designed  such  that  access  to  additional  map  detail  can  be  gained  by  zooming  in  on  an 
area  of  interest. 

Related  problems  identified  by  battle  staff  members  are  (a)  the  lack  of  large-scale  (e.g., 
1:50, 000-scale)  databases  for  some  areas,  and  (b)  the  excessive  amount  of  time  and  effort  that 
are  required  to  change  from  a  small-scale  to  a  large-scale  database.  The  S2  suggested  that  dual 
systems  should  be  provided  in  which  a  large-scale  database  is  portrayed  on  one  system  and  a 
small-scale  database  is  portrayed  on  the  other.  Most  battle  staff  members  suggest  that  more 
large-scale  databases  should  be  developed  for  use  in  both  training  and  operational  settings. 

Planning  aids.  Problems  experienced  during  planning  led  battle  staff  members  to 
recommend  system  design  improvements.  The  CO,  XO,  and  S3  expressed  dissatisfaction  with 
MCS  software.  First,  they  stated  that  the  OPORD  format  is  out  of  date.  Second,  they 
experienced  problems  in  using  the  MCS  to  create  an  execution  matrix.  Third,  they  were  unable 
to  enter  free  test  messages  at  desired  locations  in  the  execution  matrix. 
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The  CO,  XO,  and  S3  indicated  that  a  better  digital  briefing  capability  is  needed  and  that 
this  capability  should  include  a  light  pen  type  input  device  (“John  Madden  Pen”)  that  enables  the 
briefer  to  make  free-hand  annotations  on  digital  displays. 

Wargaming.  Battle  staff  members  indicated  that  the  digital  systems  do  not  provide  a 
useful  wargaming  capability  because  of  the  excessive  amount  of  time  required  to  enter  the 
requisite  data.  Although  no  specific  design  solutions  were  recommended,  battle  staff  members 
indicated  that  designers  should  give  more  thought  to  designing  the  systems  to  support 
wargaming.  Battle  staff  members  made  the  general  recommendation  that  more  automated 
wargaming  and  mission  rehearsal  tools  be  developed  for  use  in  the  ATCCS. 

CSSCS  design  improvements.  The  SI  and  S4  identified  a  host  of  problems  that,  in  their 
view,  stem  from  CSSCS  design  shortfalls.  Listed  below  are  the  design  changes  that  were 
recommended  by  the  S 1  and  S4  during  the  post-exercise  interview. 

•  The  present  CSSCS  does  not  permit  users  to  track  elements  smaller  than  a  company. 
The  system  should  be  designed  in  a  manner  that  enables  users  to  create  a  unit  with 
any  composition  of  aircraft  and  to  predict  accurately  the  fuel  and  ammunition 
consumption  of  that  element. 

•  At  present,  it  is  necessary  to  manually  enter  data  that  have  already  been  entered  in  the 
system  at  a  different  echelon.  The  system  should  be  designed  such  that  data  entered 
at  one  location  is  transferred  automatically  to  other  systems  in  the  net. 

•  Interconnectivity  among  systems  should  enable  other  system  displays  to  be  portrayed 
on  the  CSSCS  display. 

•  At  present,  the  battalion  CSSCS  is  updated  only  after  a  resupply  vehicle  has  returned 
to  the  battalion  and  the  vehicle  operator  calls  the  S4  on  the  radio.  Until  this  happens, 
the  CSSCS  shows  a  supply  shortage.  The  CSSCS  should  be  designed  to  provide 
more  timely  information  on  the  status  of  resupply  activities. 

•  The  present  system  provides  information  about  basis-of-issue  rather  than  information 
about  the  supplies  that  can  be  drawn.  The  system  should  provide  information  about 
supplies  that,  in  fact,  are  available  and  should  provide  timely  information  about 
shortages. 

•  The  system  is  not  designed  to  enable  users  to  report  or  track  casualties  effectively. 
The  system  should  be  designed  to  enable  the  user  to  track  individuals  who  were 
injured  or  killed.  Specifically,  the  system  should  enable  the  user  to  track  the  location, 
treatment,  and  condition  of  every  casualty. 

•  The  CSSCS  should  be  designed  to  provide  more  current  information  about  each 
aircraft’s  fuel  and  ammunition  load. 

Discussion  and  Conclusions 
Tactics.  Techniques,  and  Procedures 

The  requirements  that  served  as  the  impetus  for  this  study  included  the  development  of 
baseline  TTPs  for  utilizing  C4I  equipment  to  (a)  perform  the  MDMP  steps  and  (b)  monitor  the 
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execution  of  the  DST.  Necessary  constraints  on  the  design  of  this  study  made  it  unsuitable  for 
identifying  near-optimal  TTPs.  The  lack  of  comprehensive  prior  training  on  the  functions  and 
operations  of  the  digital  systems  is  among  the  most  important  of  these  constraints.  Because  of 
the  small  amount  of  training  received  before  the  study  commenced,  the  experience  that  battle 
staff  members  gained  during  the  study  was  barely  adequate  for  them  to  acquire  rudimentary 
skills  in  using  the  digital  systems.  This  problem  was  compounded  by  the  schedule  that  was 
established.  The  time  required  to  plan  and  execute  the  scheduled  missions  left  no  time  for  battle 
staff  members  to  develop  and  assess  alternate  TTPs  for  utilizing  the  digital  C4I  systems  during 
mission  planning  or  mission  execution. 

An  equally  important  constraint  is  the  limited  number  of  digital  systems  that  were 
operational  during  the  study.  It  cannot  be  assumed  that  TTPs  established  with  only  three  digital 
systems  operational  (ASAS,  MCS,  and  CSSCS)  would  be  suitable  for  use  with  a  folly 
operational  suite  of  digital  systems.  The  data  compiled  during  the  study  left  no  doubt  that  the 
procedures  that  battle  staff  members  employed  during  mission  execution  were  influenced  by  the 
lack  of  information  from  the  three  systems  that  were  not  operational  (AFATDS,  AMDW/S,  and 
FAADEO).  It  seems  certain  that  quite  different  procedures  would  have  evolved  if  the  AFATDS, 
AMDW/S,  and  FAADEO  had  been  operational  and  had  been  manned  by  skilled  operators. 

Despite  the  constraints  described  above,  this  study  yielded  useful  information  about  (a) 
problems  that  can  be  solved,  at  least  in  part,  by  developing  TTPs  and  (b)  the  general  types  of 
TTPs  that  are  needed  to  solve  the  problems.  Insights  gained  about  the  need  for  mission  planning 
TTPs  and  mission  execution  TTPs  are  discussed  below. 

Mission  Planning  TTPs 

Two  related  problems  encountered  during  mission  planning  point  to  the  need  for  TTPs 
that  would  govern  battle  staff  members’  mission  planning  activities.  One  problem  is  inadequate 
planning  time,  and  the  other  is  the  unequal  distribution  of  planning  tasks  among  members  of  the 
battle  staff.  The  problems  are  related,  because  the  planning  time  inadequacy  was  due  in  large 
part  to  the  unequal  distribution  of  workload  among  members  of  the  battle  staff.  In  short,  the 
findings  of  this  study  support  the  conclusion  that  there  is  a  need  to  develop  TTPs  that  promote  a 
more  equal  distribution  of  the  MDMP  substeps  among  battle  staff  members.  The  TTPs  must 
specify  the  distribution  of  work  among  battle  staff  members  for  normal  conditions  and  for 
conditions  in  which  the  planning  process  must  be  compressed  in  time. 

Because  the  amount  of  planning  time  can  vary  over  a  wide  range,  the  TTPs  that  govern 
planning  activities  under  a  compressed  time  schedule  should  specify  methods  for  tailoring 
planning  activities  to  the  time  that  is  available.  In  addition,  the  TTPs  should  specify  (a)  the 
degree  to  which  MDMP  substeps  can  be  performed  concurrently  and  (b)  the  battle  staff  member 
who  is  responsible  for  determining  when  work  on  an  MDMP  step/substep  must  be  terminated. 

As  was  stated  earlier,  the  number  of  substeps  assigned  by  MDMP  doctrine  dictates  an 
unequal  distribution  of  workload  among  battle  staff  members.  For  this  reason,  it  is  doubtful  that 
suitable  TTPs  can  be  developed  without  corresponding  changes  in  the  MDMP  doctrine.  It 
should  also  be  acknowledged  that  a  more  even  distribution  of  planning  activities  may  not  be 
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possible  without  increasing  the  amount  of  cross  training  of  battle  staff  members  or  increasing  the 
size  of  the  planning  staff. 

In  addition  to  TTPs  that  will  promote  a  more  equal  distribution  of  workload  among  the 
planning  staff,  TTPs  are  needed  that  specify  the  manner  in  which  each  digital  system  should  be 
used  in  the  accomplishment  of  each  MDMP  step.  The  TTPs  must  take  into  account  both  the 
overlapping  capabilities  and  the  unique  capabilities  of  the  different  digital  systems.  The  TTPs 
also  must  take  into  account  the  need  to  convey  information  from  one  system  to  another  and  the 
interactivity  (or  lack  of  interactivity)  among  the  digital  systems. 

There  are  no  reasons  to  doubt  that  inadequate  training  on  the  digital  systems  contributed 
to  many  of  the  mission  planning  problems  that  were  observed  during  this  study.  However,  it 
seems  highly  unlikely  that  such  problems  can  be  solved  through  additional  training  on  the 
capabilities  and  operation  of  each  digital  system.  For  instance,  battle  staff  members  can  be 
trained  to  perform  intervisibility  analyses  on  the  AMPS,  the  ASAS,  and  the  MCS,  but  TTPs  are 
required  to  specify  the  circumstances  in  which  each  system  should  be  used  and  by  whom  it 
should  be  used.  Similarly,  battle  staff  members  can  be  trained  on  the  interactivity  among 
systems,  but  TTPs  are  required  to  specify  the  circumstances  in  which  information  must  be 
conveyed  from  one  system  to  another  and  by  whom. 

Problems  that  battle  staff  members  encountered  during  mission  planning  support  the 
conclusion  that  TTPs  are  needed  to  improve  the  management  of  graphic  information.  As 
discussed  earlier,  battle  staff  members  reported  that  there  is  a  need  for  TTPs  that  specify  how 
best  to  (a)  identify  the  command  level  at  which  graphics  were  created,  (b)  deconflict  graphics 
created  at  different  command  levels,  and  (c)  process  changes  in  graphics.  Although  some 
procedures  were  suggested  by  the  battle  staff  members  (e.g.,  a  free  text  message  attached  to  each 
graphics  file),  more  study  is  needed  to  determine  whether  or  not  it  is  possible  to  develop 
effective  graphics  management  TTPs  without  changes  in  the  digital-system  hardware,  software, 
or  both. 

Mission  Execution  TTPs 


The  analysis  of  voice  communication  events  provides  compelling  evidence  of  two  closely 
related  information  management  problems.  First,  the  workload  associated  with  processing 
incoming  and  outgoing  information  was  often  very  high  and  was  not  evenly  distributed  among 
battle  staff  members.  When  the  volume  of  communication  events  was  highest,  the  CO  usually 
was  the  originator  and  the  recipient  of  the  vast  majority  of  communication  events.  Second,  many 
instances  were  observed  in  which  a  battle  staff  member  (usually  the  BC)  was  the  recipient  of 
important  information  that  was  not  overtly  conveyed  to  the  CO.  These  findings  support  the 
conclusion  that  there  is  a  pressing  need  for  TTPs  that  specify  information  management 
procedures  and  responsibilities. 

The  overriding  objective  of  the  information  management  TTPs  should  be  to  provide  the 
CO  with  the  information  needed  to  maintain  a  high  level  of  situation  awareness  while  keeping 
the  CO’s  information  processing  workload  at  a  manageable  level.  It  has  been  suggested  that  an 
“information  manager”  is  needed  to  unburden  the  CO  and  perhaps  other  members  of  the  battle 
staff  as  well.  However,  as  is  discussed  in  a  later  subsection,  this  study  provided  no  evidence  that 
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adding  an  “information  manager”  to  the  battle  staff  is  the  best  way  to  improve  the  management 
of  information  during  mission  execution.  Contemporary  information  theory  (Kahan  and  Worley, 
1989;  Schmitt  et  al.,  1994)  and  the  findings  of  this  study  support  the  conclusion  that  improved 
information  management  can  best  be  achieved  by  establishing  TTPs  that  govern  the  information 
management  activities  and  responsibilities  of  existing  battle  staff  members.  Although  additional 
study  is  required  to  develop  effective  information  management  TTPs,  it  seems  certain  that  the 
TTPs  must  promote  the  following: 

•  ensure  that  the  responsibility  for  monitoring  information  systems  (including  but  not 
limited  to  voice  communication  systems)  is  distributed  as  evenly  as  possible  among 
members  of  the  battle  staff, 

•  ensure  that  incoming  information  is  screened  by  the  battle  staff  members  who  are 
responsible  for  monitoring  the  information  systems, 

•  ensure  that  the  CO’s  information  needs  are  anticipated  by  the  battle  staff  members 
who  are  responsible  for  monitoring  the  information  systems,  and 

•  ensure  that  only  the  information  that  the  CO  needs  to  remain  situationally  aware  is 
transmitted  (pushed)  to  the  CO. 

Both  the  CO  and  XO  reported  that  the  information  they  needed  often  was  not  displayed 
on  the  large-format  (wall  mounted)  display  or  was  obscured  by  a  pull-down  menu  or  by  another 
window  (e.g.,  message  window).  It  often  was  necessary  for  the  CO  or  XO  to  spend  time 
instructing  operators  to  change  the  display,  eliminate  pull-down  windows,  scroll  to  a  different 
area,  or  zoom  in/out  on  an  area.  These  problems  support  the  conclusion  that  TTPs  are  needed  to 
govern  the  following: 

•  selecting  the  system  display  (ASAS,  MCS,  etc.)  that  appears  on  the  large  format 
display, 

•  displaying  pull-down  menus  and  other  windows, 

•  scrolling  to  the  area  that  is  of  interest  to  the  CO,  and 

•  displaying  the  imagery  at  the  most  suitable  magnification  level. 

The  problems  that  battle  staff  members  encountered  in  managing  graphics  information 
during  mission  planning  (see  above  description)  also  were  encountered  during  mission 
execution.  Hence,  it  is  concluded  that  TTPs  governing  graphic  management  must  be  suitable  for 
use  during  both  mission  planning  and  mission  execution.  As  stated  above,  the  TTPs  must 
establish  methods  for  identifying  the  command  level  at  which  graphics  were  created, 
deconflicting  graphics,  and  processing  changes  to  graphics.  For  mission  execution,  it  is 
particularly  important  that  the  TTPs  specify  methods  for  alerting  personnel  at  all  command 
levels  that  a  change  in  graphics  has  been  made.  Because  of  the  high  workload  imposed  on  most 
battle  staff  members  during  mission  execution,  it  is  unrealistic  to  expect  that  changes  in  graphics 
would  be  detected  in  a  reliable  and  timely  manner  without  an  alerting  cue  of  some  type. 

TTPs  for  Monitoring  Execution  of  DST 

One  of  the  requirements  listed  in  the  introductory  section  of  this  report  was  to  develop 
TTPs  for  monitoring  the  execution  of  the  DST  utilizing  digital  C4I  systems.  This  study  yielded 
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little  information  that  bears  on  this  requirement.  Although  a  DST  was  developed  during  the 
planning  of  each  mission,  there  was  no  evidence  that  the  CO  or  any  other  battle  staff  member 
monitored  the  execution  of  the  DST  during  the  execution  of  any  mission.  No  overt  actions  or 
comments  were  observed  indicating  that  the  CO’s  review  of  options  or  his  decisions  were  keyed 
to  the  location  of  friendly  or  enemy  forces  relative  to  pre-planned  decision  points  (DPs),  target 
areas  of  interest  (TAIs),  named  areas  of  interest  (NAIs),  or  time-phase  lines. 

This  study  yielded  no  specific  information  about  the  problems  that  battle  staff  members 
encounter  when  monitoring  the  execution  of  a  DST.  However,  other  information  gained  from 
the  study  can  be  used  to  speculate  about  the  general  design  requirements  for  TTPs  that  would 
remediate  problems  that  are  likely  to  be  encountered.  The  requirements  discussed  below  follow 
from  the  assumptions  that  the  TTPs  must  be  designed  to  minimize  the  information  processing 
burden  imposed  on  the  battle  staff,  especially  the  information  processing  burden  imposed  on  the 
CO. 


First,-  the  TTPs  that  govern  DST  monitoring  should  minimize  the  requirements  for  battle 
staff  members  to  remember  the  location  of  DPs  and  the  events  that  trigger  the  CO  to  review 
options  or  to  make  decisions.  This  could  be  accomplished  by  producing  a  special  graphic 
overlay  (for  one  or  more  of  the  digital  systems)  that  depicts  all  or  most  of  the  information 
contained  on  a  conventional,  hard-copy  DST. 

Second,  the  TTPs  should  ensure  that  responsibility  for  monitoring  the  execution  of  the 
DST  is  shared  by  two  or  more  battle  staff  members.  It  is  well  known  that  monitoring  tasks  are 
performed  poorly  by  human  operators,  especially  when  they  must  perform  other  information 
processing  tasks  concurrently.  It  is  for  this  reason  that  the  TTPs  should  ensure  that  monitoring 
the  execution  of  the  DST  is  not  the  sole  responsibility  of  one  battle  staff  member. 

Third,  the  TTPs  should  ensure  that  monitoring  the  execution  of  the  TTP  does  not  impose 
an  excessive  workload  on  the  battle  staff  members  to  whom  the  monitoring  task  is  assigned. 

One  approach  to  meeting  this  requirement  is  to  relieve  the  battle  staff  members  of  all 
responsibilities  other  than  monitoring  the  execution  of  the  DST.  Another  approach  is  to 
subdivide  the  monitoring  task  into  elements  and  distribute  the  elements  among  two  or  more 
battle  staff  members.  For  example,  the  FSO  and/or  S2  could  be  given  responsibility  for 
monitoring  the  DPs  that  trigger  supporting  fires.  Similarly,  the  SI  and  S4  could  be  given 
responsibility  for  monitoring  the  DPs  that  trigger  the  displacement  of  a  support  area  or  the 
delivery  of  supplies.  If  the  monitoring  task  is  subdivided  in  this  manner,  it  may  be  desirable  to 
tailor  a  unique  decision  support  graphic  for  each  battle  staff  member  who  has  responsibility  for 
monitoring  and  for  the  digital  system  used  by  the  battle  staff  member. 

Finally,  the  TTPs  should  ensure  that  battle  staff  members  who  monitor  the  execution  of 
the  DST  alert  the  CO  in  a  timely  manner  when  events  have  occurred  that  trigger  option 
evaluation  and  decision  making.  The  TTPs  must  establish  rules  that  specify  when  the  CO  is  to 
be  alerted.  The  rules  may  be  defined  in  terms  of  elapsed  time,  the  proximity  of  friendly  or 
enemy  forces  to  a  specified  location,  the  reaction  of  enemy  forces  to  a  friendly  attack,  and  so  on. 
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The  TTP  requirements  outlined  above  assume  that  the  monitoring  of  the  execution  of  the 
DST  must  be  accomplished  manually.  It  is  possible  that  one  or  more  of  the  digital  C4I  systems 
could  be  programmed  to  signal  the  operator  when  events  have  occurred  that  trigger  a  DP.  No 
attempt  has  been  made  to  assess  the  feasibility  of  automating  all  or  a  part  of  the  DST  execution 
monitoring  function. 

Administrative  TTPs 

Problems  reported  by  battle  staff  members  support  the  conclusion  that  TTPs  are  needed 
to  accomplish  the  following: 

•  ensure  that  all  information  from  prior  missions  has  been  removed  from  the  digital 
systems, 

•  ensure  that  all  echelons  employ  a  common  file-naming  method  and  a  common  date¬ 
time  labeling  method,  and 

•  ensure  that  all  files  transferred  among  systems  or  echelons  are  accompanied  by  a  free 
text  message  that  describes  the  contents  of  the  file. 

Need  for  an  Information  Manager 

One  of  the  requirements  for  this  study  was  to  determine  whether  or  not  an  information 
manager  is  needed  in  the  ATCCS  and,  if  so,  who  that  person  should  be.  Battle  staff  members 
were  questioned  about  the  need  for  an  information  manager  through  questionnaire  items  and 
post-exercise  interviews.  Although  nearly  one-half  of  the  battle  staff  members  indicated  that  an 
information  manager  is  needed,  they  held  widely  different  views  about  the  function  of  an 
information  manager. 

The  most  common  function  mentioned  was  that  of  a  technical  expert  who  would  keep  the 
digital  systems  running  and  would  provide  direct  support  in  operating  the  systems.  The  findings 
of  this  study  support  the  conclusions  that  (a)  several  individuals  are  needed  to  serve  as  digital 
system  maintainers  and  operators,  and  (b)  an  individual  is  needed  to  monitor  the  situation  and  to 
pass  information  about  the  friendly  situation  and  enemy  situation  along  to  the  battle  staff 
members  who  are  engaged  in  mission  planning. 

This  study  provided  no  evidence  to  support  the  conclusion  that  it  would  be  feasible  or 
beneficial  to  add  a  battle  staff  member  who  would  be  solely  responsible  for  evaluating  incoming 
information  and  deciding  who  should  receive  the  information.  The  CO  stated  that  information 
management  is  one  of  the  CO’s  primary  responsibilities.  However,  both  the  CO  and  XO 
expressed  the  view  that  digital  system  operators  could  be  and  should  be  trained  to  screen 
incoming  information  and  to  “push”  important  information  to  the  CO. 

Configuration  of  C4I  System  Displays 

Among  the  requirements  that  were  the  impetus  for  this  study  was  the  requirement  to 
“develop  a  recommended  C4I  system  display  configuration  which  best  supports  the  commander’s 
information  requirements.”  Because  of  the  constraints  discussed  above,  it  was  not  possible  to 
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develop  and  assess  alternative  C4I  system  display  configurations  during  this  study.  The  only 
information  compiled  during  this  study  that  bears  on  this  requirement  is  a  list  of  the  problems 
that  battle  staff  members  encountered  in  using  the  display  formats.  Hence,  inferences  about  a 
near-optimal  C4I  system  display  configuration  must  be  drawn  from  a  consideration  of  (a)  the 
problems  that  battle  staff  members  encountered  during  this  study  and  (b)  the  human  factors 
design  principles  that  apply  to  information  displays. 

Described  below  are  digital  C4I  display  system  requirements  that  were  inferred  from  the 
information  compiled  during  this  study  and  from  relevant  human  factor  design  principles.  Some 
requirements  extend  beyond  the  formatting  of  information  on  digital  C4I  system  displays,  but  all 
are  aimed  at  providing  the  information  that  the  CO  needs  to  establish  and  maintain  a  high  level 
of  situation  awareness  during  the  execution  of  a  mission. 

Display  Resolution 

As  was  discussed  earlier,  there  were  many  instances  in  which  the  battle  staff  members 
were  unable  to  discriminate  important  information  that  was  portrayed  on  the  large-format  wall 
mounted  display.  Close  inspection  by  members  of  the  research  staff  led  to  the  conclusion  that 
the  single  most  important  reason  for  the  inadequate  discriminability  was  the  fact  that 
alphanumeric  characters  and  point  symbols  were  depicted  with  too  few  raster  lines.  However, 
inadequate  height  of  some  symbols  and  inadequate  figure  ground  contrast  also  contributed  to 
problems  in  discriminating  characters  and  point  symbols. 

Research  has  shown  that  a  character  height  of  about  20  min  of  arc  portrayed  with  10 
raster  lines  (per  symbol  height)  is  required  to  achieve  a  character  resolution  accuracy  of  about 
80%  (Baker  and  Nicholson,  1967;  Hemingway  and  Erickson,  1969).  The  requirements  for 
symbol  subtense  and  raster  lines  per  symbol  height  are  greater  for  some  symbols  and  for  off-axis 
viewing  of  the  display.  -  It  should  also  be  noted  that  the  minimum  character  height  and  the 
minimum  number  of  raster  lines  per  character  height  vary  as  a  function  of  character  stroke  width, 
display  brightness,  brightness  contrast,  color  contrast,  and  various  types  of  display  aberrations. 
The  relationships  between  such  display  design  variables  and  display  legibility  are  described  in 
human  factors  design  handbooks  and  engineering  data  handbooks  (for  example,  see  Boff  and 
Lincoln,  1988). 

The  development  of  display  design  specifications  to  insure  adequate  legibility  of  the  C4I 
system  displays  that  are  to  be  used  by  the  CO  and  other  battle  staff  members  is  beyond  the  scope 
of  this  study.  Such  an  effort  would  require  an  assessment  of  alternate  display  types  (e.g.,  head- 
mounted  displays,  desk-mounted  displays,  wall-mounted  displays)  and  a  human  factors  design 
requirements  analysis  for  the  display  type  that  is  judged  optimal.  The  human  factors  design 
requirements  analysis  must  be  based  on  a  consideration  of  the  viewing  distance,  the  ambient 
lighting  conditions,  display  vibration,  and  the  full  range  of  characters  and  symbols  to  be 
displayed.  / 
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Size  of  Point  and  Linear  Symbols 

Battle  staff  members  reported  that  the  linear  symbols  depicted  on  the  displays  often  were 
so  large  that  they  obscured  an  excessive  amount  of  map  detail.  Although  no  battle  staff  member 
reported  that  point  symbols  were  too  large,  it  seems  certain  that  some  of  the  point  symbols 
depicted  on  the  AS  AS  and  MCS  were  large  enough  to  obscure  important  map  detail.  Research  is 
needed  to  assess  at  least  three  approaches  to  reducing  the  obscuration  of  map  detail  by  point  and 
linear  symbols.  The  research  should  assess  the  feasibility  of  using  these  approaches  singly  or  in 
combination  to  encode  and  portray  linear  and  point  symbols  on  a  C4I  system  display. 

The  most  obvious  approach  is  to  simply  reduce  the  width  of  linear  symbols  and  reduce 
the  diameter  of  point  symbols.  The  amount  that  symbol  size  can  be  reduced  and  still  be  easily 
legible  depends  on  all  of  the  factors  discussed  above  that  influence  display  resolution.  The  width 
of  linear  symbols  often  can  be  reduced  by  a  substantial  amount  and  still  remain  legible. 

However,  a  lessor  reduction  in  width  is  possible  if  the  linear  symbol  must  be  discriminated  from 
another  linear  symbol  (e.g.,  a  dotted  line  vs.  a  dashed  line).  The  same  comments  apply  to  point 
symbols.  A  very  small  point  symbol  can  be  detected,  but  a  larger  size  is  required  if  it  is 
necessary  to  discriminate  among  point  symbols  (e.g.,  a  circle  vs.  a  triangle). 

A  second  approach  is  to  color  code  point  and  linear  symbols  rather  than  shape  code  them. 
Very  narrow  linear  symbols  can  be  differentiated  if  they  are  depicted  in  a  different  color.  An 
even  greater  size  reduction  in  point  symbols  is  possible  if  they  are  color  coded  rather  than  size 
coded. 


A  third  approach  is  to  provide  the  operators  with  the  capability  to  temporarily  remove 
linear  symbols  or  point  symbols  that  obscure  important  map  detail.  Ideally,  the  digital  systems 
would  be  programmed  to  enable  operators  to  temporarily  remove  a  single  symbol,  all  symbols  of 
a  specified  class,  or  all  symbolic  information  depicted  on  a  C4I  system  display. 

Map  Scale  and  Map  Detail 

Throughout  a  mission,  the  CO  frequently  changed  the  size  of  the  area  that  appeared  on 
the  situation  display  that  he  was  using  (usually  the  MCS).  Similarly,  the  map  detail  that  was  of 
interest  to  the  CO  varied  throughout  a  mission.  The  CO’s  need  for  a  larger  or  smaller  viewing 
area  was  nearly  always  closely  related  to  the  level  of  map  detail  that  was  of  interest  to  him. 

When  he  was  interested  in  viewing  a  very  small  area  of  the  battlefield,  the  CO  was  interested  in 
the  type  of  map  detail  that  appears  on  a  large-scale  map.  When  he  was  interested  in  viewing  a 
large  area  of  the  battlefield,  the  CO  had  no  interest  in  the  map  detail  that  appears  on  a  large-scale 
map.  Rather,  he  was  interested  in  getting  a  clear  picture  of  the  location  of  friendly  forces  and 
enemy  forces  relative  to  one  another  and/or  relative  to  large  topographic  features. 

When  the  topographic  database  remained  the  same,  zooming  in  on  a  small  area  did  not 
increase  the  amount  of  map  detail  that  was  visible.  Zooming  in  did  nothing  more  than  magnify 
the  size  of  map  features,  which  usually  could  be  discriminated  without  zooming  in  on  them. 
Access  to  greater  map  detail  was  possible  only  by  changing  the  database  to  one  that  was 
compiled  at  a  larger  scale,  a  change  that  is  often  too  time  consuming. 
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The  problems  encountered  by  the  CO  and  other  battle  staff  members  point  to  the  need  for 
a  system  that  displays  progressively  more  map  detail  as  the  user  zooms  in  on  an  area  of  interest 
and,  conversely  progressively  less  map  detail  as  the  user  zooms  out.  Providing  such  a  capability 
is  not  an  easy  task.  It  would  be  necessary  to  develop  software  that  selects  different  classes  of 
topographic  features  for  portrayal  depending  on  the  size  of  the  area  portrayed.  The  software  also 
would  have  to  change  the  rules  of  topographic  feature  generalization  as  a  function  of  changes  in 
the  size  of  the  area  portrayed. 

The  S2  recommended  what  may  be  a  more  feasible  solution  to  this  problem.  He 
recommended  that  the  CO  be  provided  with  duplicate  systems  that  are  the  same  in  all  respects 
except  the  scale  of  the  topographic  database  that  is  loaded  in  the  systems.  One  system  would  be 
loaded  with  a  large-scale  database  (e.g.  1:50, 000-scale)  and  the  other  system  would  be  loaded 
with  a  smaller  scale  database  (e.g.,  1:250, 000-scale).  Although  this  solution  may  increase  the 
requirements  for  equipment,  space,  and  personnel,  it  is  technically  feasible  at  the  present  time. 

Clutter  Control 

The  CO  and  XO  reported  that  (a)  their  need  for  symbolic  information  does  not  remain 
constant  throughout  a  mission  and  (b)  portraying  all  the  symbolic  information  that  is  needed  at 
some  point  during  a  mission  results  in  excessive  display  clutter.  These  observations  point  to  the 
need  for  a  convenient  way  to  control  display  clutter  by  displaying  only  the  symbolic  information 
that  is  needed  for  the  mission  phase  that  is  underway.  Although  most  digital  systems  enable  the 
operator  to  add  or  remove  different  classes  of  symbolic  information,  it  would  be  excessively  time 
consuming  and  distracting  for  the  CO  or  another  battle  staff  member  to  evaluate  and  to  modify 
symbolic  information  throughout  a  mission.  A  logically  appealing  approach  proposed  by  the  XO 
was  to  develop  a  small  set  of  symbolic  information  formats  that,  together,  would  meet  the  CO’s 
needs  throughout  the  mission.  The  symbolic  information  formats  could  be  developed  to  fulfill 
the  information  needs  during  each  mission  phase,  during  special  situations,  or  both. 

Deaggregation  of  Friendly  and  Enemy  Unit  Icons 

The  CO  indicated  that  the  use  of  regimental  icons  to  depict  the  location  of  enemy  forces 
is  not  adequate  for  some  situations  and  for  some  users.  He  stated  that  digital  C4I  display  systems 
should  enable  the  CO  and  other  battle  staff  members  to  deaggregate  the  regiment  icons  into  the 
companies  that  comprise  the  regiment.  That  is,  icons  would  be  displayed  showing  the  locations 
of  the  companies  that  comprise  the  regiments.  Although  not  stated  by  the  CO,  it  is  assumed  that 
the  regimental  icons  would  be  preferred  under  some  circumstances.  If  this  assumption  is  correct, 
the  digital  C4I  display  systems  should  enable  the  user  to  both  aggregate  and  deaggregate  the 
icons  that  depict  the  location  of  enemy  units. 

Identification  “Tags”  for  Friendly  Aircraft  Symbols 

The  data  compiled  during  this  study  confirmed  the  well-known  fact  that  maintaining  an 
awareness  of  the  location  of  friendly  aircraft  is  a  vital  part  of  the  CO’s  situation  awareness.  It  is 
also  a  well-known  fact  that  one  of  the  most  important  requirements  of  digital  C4I  display  systems 
is  to  provide  accurate  and  current  information  about  the  location  of  individual  friendly  aircraft. 


93 


The  XO  suggested  that  the  effectiveness  of  a  system  that  tracks  and  displays  the  location  of 
friendly  aircraft  would  be  improved  dramatically  if  identification  “tags”  could  be  attached  to  the 
friendly  aircraft  icons.  There  is  no  doubt  that  the  capability  to  attach  identification  “tags”  to 
friendly  aircraft  icons  is  a  highly  desirable  requirement  for  a  digital  C4I  display  system. 

Coding  of  Threat  Icons 

Problems  reported  by  the  CO  and  XO  point  to  the  need  for  a  better  system  for  encoding 
the  threat  icons  that  depict  threat  vehicles.  One  need  is  for  the  capability  to  modify  the  icons  in  a 
way  that  depicts  the  age  of  the  intelligence  information  that  was  used  to  depict  a  threat  icon  at  a 
specific  location.  For  instance,  the  XO  suggested  that  the  color,  shape,  or  brightness  of  a  threat 
icon  could  be  changed  systematically  to  reflect  the  age  of  the  intelligence  information  that  was 
used  to  establish  the  location  of  the  threat.  A  similar  requirement  is  for  a  capability  to  employ  a 
coding  system  for  threat  icon  that  depicts  whether  the  threat  is  active  or  has  been  destroyed. 
Although  an  important  requirement  for  digital  C4I  display  systems,  research  is  needed  to 
determine  the  optimal  method  for  encoding  threat  icons  to  reflect  the  currency  of  intelligence 
information  on  which  they  are  based  and  the  status  (active  or  destroyed)  of  the  threat. 

Lessons  Learned  About  Research  Methodology 

Some  of  the  lessons  learned  about  the  methodology  used  in  this  study  are  discussed 

below. 

Prior  Training  of  Battle  Staff  Members 

The  small  amount  of  training  that  battle  staff  members  received  on  the  functions  and 
operation  of  the  digital  systems  has  been  mentioned  repeatedly  throughout  this  report.  Although 
a  necessary  constraint,  it  must  be  acknowledged  that  more  could  have  been  learned  about  how 
best  to  use  the  digital  equipment  (TTPs)  if  battle  staff  members  had  received  more  training  prior 
to  the  start  of  the  study.  In  future  studies  of  this  type,  every  attempt  should  be  made  to  acquire 
the  time  and  resources  needed  to  provide  battle  staff  members  with  more  comprehensive  training 
on  the  functions  and  operation  of  the  digital  systems. 

Realism  of  Mission  Scenarios  and  Scheduling 

For  the  most  part,  the  battle  staff  members  indicated  that  the  mission  scenarios  and  the 
scheduling  of  exercises  were  quite  realistic.  Battle  staff  members  suggested  that  realism  could 
be  improved  by  increasing  the  volume  of  intelligence  information  that  must  be  processed  during 
mission  planning  and  during  mission  execution.  The  low  volume  of  intelligence  information 
caused  the  workload  of  some  battle  staff  members  to  be  unrealistically  low.  Battle  staff 
members  also  reported  that  improvements  in  realism  could  be  achieved  by  making  the  SI  and  S4 
perform  a  more  realistic  number  of  the  tasks  they  ordinarily  perform  during  mission  planning 
and  mission  execution. 

There  were  numerous  reports  that  an  inadequate  amount  of  time  was  made  available  for 
mission  planning.  In  one  instance,  the  time  between  the  receipt  of  a  FRAGO  and  aircraft  launch 
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was  clearly  insufficient  for  adequate  mission  planning.  Other  reports  of  inadequate  planning 
time  probably  were  due  more  to  the  battle  staffs  lack  of  training  than  to  the  absolute  amount  of 
time  that  was  available  for  mission  planning. 


Identification  of  Mission  Planning  Problems/Oversights 

The  mission  planning  questionnaire  completed  by  the  battle  staff  members  did  not  prove 
to  be  an  effective  way  to  obtain  detailed  information  about  the  mission  planning  problems  and 
oversights.  Although  some  battle  staff  members  took  the  time  to  write  a  description  of  mission 
planning  problems  and  oversights,  most  did  not.  The  reluctance  to  draft  written  descriptions  of 
problems  was  not  altogether  unexpected. 

The  mission  planning  observations  proved  to  be  an  even  less  effective  method  for 
obtaining  detailed  information  about  mission  planning  problems  and  oversights.  Although  the 
mission  planning  observers  were  reasonably  knowledgeable  about  the  MDMP  steps  and 
substeps,  they  were  unable  to  identify  and  describe  the  MDMP  steps/substeps  that  were  not 
performed  adequately,  much  less  the  reasons  for  inadequate  performance. 

It  seems  unlikely  that  either  the  questionnaire  items  or  the  observation  procedures  could 
be  modified  to  provide  detailed  and  comprehensive  information  about  mission  planning 
problems/oversights.  In  future  studies  of  this  type,  consideration  should  be  given  to  face-to-face 
interviews  with  individual  battle  staff  members  as  a  means  of  obtaining  detailed  information 
about  mission  planning  problems/oversights.  Some  mission  planning  problems/oversights  do  not 
become  apparent  until  after  a  mission  has  been  executed.  Therefore,  interviews  to  identify 
specific  mission  planning  problems  and  oversights  are  likely  to  be  most  effective  if  they  are 
conducted  immediately  after  the  mission  has  been  executed. 

Identification  of  TTPs 

An  observation  study  is  not  an  effective  way  to  identify  near-optimal  TTPs  for  using 
digital  C4I  systems.  Even  if  the  battle  staff  members  had  received  comprehensive  training  on  the 
digital  systems,  this  study  would  have  yielded  information  about  only  one  set  of  TTPs,  which 
may  have  been  highly  effective  or  highly  ineffective.  Near-optimal  TTPs  can  be  identified  with 
certainty  through  an  experimental  study  in  which  performance  is  measured  as  TTPs  are  varied 
systematically.  The  only  alternative  to  an  experimental  study  is  to  convene  a  team  of  SMEs  to 
discuss  alternate  TTPs  and  to  adopt  the  TTPs  that  the  SMEs  judge  to  be  best.  Thereafter,  the 
TTPs  could  be  employed  in  training  exercises  and  refined  when  necessary. 

Another  observation  worth  noting  is  that  neither  the  battle  staff  members  nor  the 
members  of  the  research  staff  have  a  clear  understanding  of  the  meaning  of  the  term  TTPs.  No 
official  definition  of  TTPs  has  been  located  in  any  Army  document.  The  findings  of  this  study 
support  the  conclusion  that  an  official  definition  of  the  term  TTP  should  be  developed  or  use  of 
the  term  should  be  abandoned. 
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Use  of  Mission  Planning  Time  as  a  Measure  of  Proficiency 


For  many  tasks,  the  time  to  perform  the  task  is  a  valid  measure  of  task  proficiency.  The 
present  study  indicates  that  the  amount  of  time  required  to  perform  the  MDMP  steps, 
individually  or  collectively,  is  not  a  reliable  measure  of  the  proficiency  of  the  battle  staff 
members  who  participate  in  mission  planning.  The  reason  is  that  many  factors  other  than 
proficiency  influence  the  amount  of  time  spent  performing  the  MDMP  steps.  Perhaps  the  most 
important  factor  is  the  battle  staff  members’  inclination  to  always  use  all  of  the  planning  time 
that  is  available.  Another  factor  is  that  the  characteristics  of  the  mission  being  planned  have  an 
enormous  influence  in  the  amount  of  time  that  is  required  to  plan  the  mission.  For  these  reasons 
and  others,  the  time  spent  planning  a  mission  cannot  be  expected  to  serve  as  a  valid  predictor  of 
the  proficiency  of  the  mission  planning  staff. 

Improvements  in  Mission  Execution  Observations 

This  study  showed  that  passive  observation  by  relatively  inexperienced  observers  can 
yield  a  great  deal  of  useful  information  about  the  activities  and  problems  that  occur  during 
mission  execution.  However,  there  are  ways  to  improve  the  information  that  is  recorded  by  the 
observers.  One  way  is  to  replace  the  voice  communication  system  used  in  this  study  with  a 
system  in  which  headphones  rather  than  speakers  are  used.  With  speakers,  it  is  impossible  for 
observers  to  track  the  flow  of  information  to  and  from  each  battle  staff  member.  That  is,  it  is 
impossible  for  observers  to  determine  the  battle  staff  members  who  hear  messages  that  are 
announced  over  the  speakers. 

Additional  training  of  observers  is  needed  to  improve  their  ability  to  recognize  key  events 
and  problems  that  occur  during  mission  execution.  For  example,  additional  training  probably 
would  increase  observers’  ability  to  recognize  and  record  key  decision  points  that  occurred 
during  the  mission  and  instances  of  heavy  workload.  However,  it  seems  unlikely  that  additional 
training  would  significantly  improve  observers’  ability  to  recognize  decision  errors,  problems' 
encountered  in  operating  the  digital  equipment,  and  problems  encountered  in  interpreting  the 
digital  system  displays.  With  few  exceptions,  the  information  needed  to  recognize  such 
problems  is  not  available  to  a  passive  observer.  These  findings  support  the  conclusion  that  other 
methods  are  needed  to  identify  the  various  problems  that  battle  staff  members  encounter  during 
mission  execution. 

Analysis  of  Communication  Events 

The  findings  of  this  study  support  the  conclusion  that  the  analysis  of  voice 
communication  events  yields  valuable  information  about  information  management  problems,  the 
absolute  and  relative  workload  of  battle  staff  members,  and  the  command  style  of  the  CO. 
Although  very  labor  intensive,  the  recording  and  analysis  of  communication  events  is  highly 
recommended  for  future  research  on  the  use  of  digital  C4I  systems. 

It  is  also  recommended  that  time  and  resources  are  invested  to  develop  equipment  to 
automate  all  or  a  part  of  communication  event  recording  and  analysis.  The  automation  of 
communication  event  recording  and  analysis  would  greatly  increase  the  effectiveness  of  future 
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research  on  the  use  of  digital  C4I  systems.  Of  equal  importance,  automation  of  communication 
event  recording  and  analysis  would  enable  training  personnel  to  provide  feedback  that  should 
enhance  trainees’  information  management  techniques  and  their  command  style.  Such  feedback 
should  be  of  enormous  value  for  training  digital  system  operators  to  screen  incoming  information 
and  to  “push”  relevant  information  to  the  CO. 
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APPENDIX  A 

INFORMED  CONSENT  AGREEMENT 
AND 

DEMOGRAPHIC  QUESTIONNAIRE 

COMPLETED  BY  ALL  INDIVIDUALS  WHO  SERVED  AS  A  MEMBER  OF  THE 

BATTLE  STAFF  IN  THE  ATCCS 
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DIGITIZED  TOC  STUDY 

Army  Tactical  Command  and  Control  System  in  the  Aviation  Battalion  Tactical  Operations  Center 

US  ARMY  RESEARCH  INSTITUTE  ROTARY  WING  AVIATION  RESEARCH  UNIT, 

FORT  RUCKER,  ALABAMA 


CONSENT  FORM 


INTRODUCTION :  This  project  is  part  of  a  Training  and  Doctrine  Command  (TRADOC)  sponsored  one-year  project 
conducted  under  the  Army  Research  Institute  (ARI)  Study  and  Analysis  Program,  Work  Package  2151.  The  study  is 
titled  FORCE  XXI  AVIATION  BATTLE  STAFF  TRAINING:  INTEGRATION  OF  C4I  INTO  BATTLE ' 
SIMULATION.  The  major  objective  of  the  study  is  to  define  methods  by  which  lessons  learned  from  Advanced 
Concept  Technology  Demonstrations,  Advanced  Warfighting  Experiments,  and  other  exercises  relating  to  training  and 
operations  issues  of  Command,  Control,  Communications  and  Computer  Integration  (C4I),  may  be  incorporated  into 
battle  staff  training  more  rapidly  through  constructive  and  virtual  simulation.  Within  the  ATCCS  simulation  exercise 
the  digitized  C4I  system  will  be  operated  to  discover  opportunities  for  improvements  to  information  management  for 
decision  making.  Emphasis  will  be  on  the  processes  for  developing  a  tactical  Course  Of  Action  and  Decision  Support 
Template.  It  will  also  be  evaluated  to  define  and  designate  an  operations  information  management  role. 

DISCLOSURE:  Your  activities  will  include  functioning  as  a  member  of  an  Aviation  Battalion  Battle  Staff  during 
simulated  Brigade  operations.  We  will  measure  various  aspects  of  your  performance  in  the  simulator  during  the  tactical 
scenarios.  You  will  be  asked  to  complete  a  questionnaire  that  will  record  information  regarding  your  background, 
training  and  work-related  experience.  In  addition,  you  will  be  asked  to  complete  other  survey  type  forms  to  record  your 
experiences  with  the  ATCCS  and  your  impressions  regarding  training  benefits. 

RISK:  No  physical  or  emotional  risks  have  been  identified  in  this  research  protocol.  The  level  of  stress  generated  by 
participation  in  this  research  is  expected  to  be  minimal.  There  are  no  hidden  measures  or  hidden  purposes  within  this 
study.  Nor  is  there  any  deception  used  in  this  research  protocol. 

CONFIDENTIALITY :  All  information  will  be  kept  in  strictest  confidence.  Only  group  summary  results  will  ever  be 
discussed  or  reported.  No  personally  identifiable  information  will  be  used  in  reporting  results  of  this  project  to  any 
agency,  either  within  or  outside  the  US  Army.  Individuals  and  units  participating  in  this  research  will  remain 
anonymous.  YOU  HAVE  THE  RIGHT  TO  REFUSE  TO  PROVIDE  ANY  OR  ALL  INFORMATION  WITHOUT 
RISK  OF  ANY  NEGATIVE  CONSEQUENCE  TO  YOU.  This  right  is  protected  under  provisions  oL4fl  70-25  Use  of 
Volunteers  as  Subjects  of  Research. 

ACKNOWLEDGMENT :  By  signing  below  I  acknowledge  that  I  have  been  informed  that  I  have  the  right  to  refuse  to 
provide  any  or  all  information  asked  of  me.  I  further  acknowledge  that  I  have  been  informed  that  any  and  all 
information  that  I  choose  to  provide  will  be  kept  anonymous. 


Signature  Date 

POINT  OF  CONTACT:  Dr.  William  R.  Howse 

ARI-RWARU 
PERI-ER 
Bldg.  5100 

Fort  Rucker  AL  36362-5354 
334-255-368 6  DSN  558-3686 

[ARI-RWARU  KEEPS  ONE  COPY.  PARTICIPANT  KEEPS  ONE  COPY.] 
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Participant  Questionnaire 


The  purpose  of  this  questionnaire  is  to  obtain  information  about  the  training  and 
experience  that  you  bring  to  Digital  Tactical  Operations  Center  (TOC)  Study. 
Information  about  the  training  and  experience  of  participants  is  essential  for  establishing 
the  credibility  of  the  research  findings,  conclusions,  and  recommendations. 


1 .  Name _ 

2.  Age _  3.  Current  Grade _  4.  Duty  Phone 

5.  Unit  Address  _ _ 

6.  What  is  your  assigned  battle  staff  position(s)  in  this  study? _ 


7.  What  is  your  Specialty  Skill  Identifier  (SSI)  or  your  Primary  Occupational  Specialty 
(PMOS)? 

SSI _ 

PMOS _ 

6.  How  many  years  of  military  service  do  you  have? 

*  _ years  Active  Component  Service 

_ years  Reserve  Component  Service 

7.  What  is  your  current  duty  assignment? 


8.  How  long  have  you  served  in  your  present  duty  assignment? 
months 


9.  List  the  military  education  courses  you  have  completed. 


10.  List  the  military  technical  schools  you  have  completed. 


11.  List  the  number  of  months  you  have  served  in  each  of  the  following  battle  staff 
positions.  (Enter  “0”  if  none.) 

_  months  experience  as  battalion  commander 

_  months  experience  as  battalion  XO 

_  months  experience  as  battalion  SI 

_  months  experience  as  battalion  S2 

_____  months  experience  as  battalion  S3 

_  months  experience  as  battalion  S4 

_  months  experience  as  battalion  FSO 

_ _  months  experience  as  battalion  ADO 

_  months  experience  as  battalion  UAV  operator 

______  months  experience  as  battalion  JSTARS  operator 

_____  months  experience  as  battalion  ASAS  operator 

_  months  experience  as  battalion  AFATDS  operator 

_ months  experience  as  battalion  MCS/P  operator 

______  months  experience  as  battalion  AMDWS  operator 

months  experience  as  battalion  FAADEO  operator 

12.  Indicate  your  level  of  knowledge  about  the  functional  capabilities  of  each  of  the 
following  digital  system. 


High 

Moderate 

None 

JSTARS 

[] 

[  ] 

[  3 

[  3 

[  3 

ASAS 

[  ] 

[  ] 

[  3 

[  3 

[  3 

AFATDS 

[  ] 

[  ] 

[] 

[  3 

[  3 

MCS/P 

[  ] 

[  ] 

[  3 

[3 

[  3 

FAADEO 

[] 

[  3 

[  3 

[  3 

[  3 

13.  Indicate  your  level  of  skill  in  operating  of  each  of  the  following  digital  system. 


High 

Moderate 

None 

JSTARS 

[  3 

[  3 

[  3 

[  3 

[  3 

ASAS 

[  3 

[  3 

[  3 

[  3 

[  3 

AFATDS 

[  3 

[  3 

[  3 

[  3 

[  3 

MCS/P 

[  3 

[  3 

[] 

[  3 

[  3 

FAADEO 

[  3 

[  3 

[  3 

[  3 

[  3 
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Mission  Planning  Questionnaire 

Name _ _ _ _  Today’s  Date _ 

Your  battle  staff  position  in  today’s  exercise  _ _ _ 

1.  About  how  much  time  did  you  spend  planning  today’s  mission? 

_ hours  and  __  minutes 

2.  How  difficult  was  it  to  plan  this  mission  compared  with  an  operational  mission  of  the 
same  type?  (check  one) 

[  ]  much  easier 
[  ]  somewhat  easier 
[  ]  about  the  same 
[  ]  somewhat  more  difficult 
[  ]  much  more  difficult 
[  ]  don’t  know 

3.  Describe  ways  in  which  the  planning  of  today’s  mission  was  more  difficult  and  the 
ways  it  was  less  difficult  than  the  planning  of  an  operational  mission  of  the  same 
type. 

Ways  more  difficult  _ _ - _ _ _ 


Ways  less  difficult 


4.  Please  rate  the  realism  of  today’s  mission  compared  with  a  similar  operational 
mission. 

[  ]  highly  unrealistic 
[  ]  moderately  unrealistic 
[  ]  highly  realistic 
[  ]  don’t  know 

Describe  the  ways  in  which  today’s  mission  was  at  least  moderately  unrealistic 
(compared  with  a  similar  operational  mission). 
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The  following  section  includes  items  that  ask  you  to  rate  the  adequacy  of  the  time 
available  and  information  available  to  perform  each  step  in  the  Military  Decision  Making 
Process  (MDMP).  This  section  also  includes  items  that  ask  you  about  the  problems  you 
encountered  in  performing  these  steps  and  potential  solutions  to  these  problems.  When 
responding  to  these  items,  please  refer  to  the  attached  list  to  jog  your  memory  about  the 
general  and  specific  steps  in  the  MDMP. _ _ _ _ _ 


5. 


Please  rate  the  adequacy  of  the  time  available  to  perform  each  of  the  following  steps 
of  the  MDMP. 


Adequacy  of  Time 


General  Steps 

Adequate 

Inadequate 

Receive  the  Mission 

[  3 

[  3 

Analyze  the  Mission 

[  3 

[  3 

Development  of  COA 

[  3 

[  3 

Analyze  the  COA  (War  Game) 

[  3 

[  3 

Compare  the  COA 

[  3 

[  3 

Approve  the  COA 

[  3 

[  3 

Produce  the  Orders 

[  3 

[  3 

Did  Not  Perform 

[] 

[  ] 

[  ] 

[  ] 

[  1 
[  ] 

[  ] 


6. 


Plpasf.  rsitfi  the  adequacv  of  the  information  available  to  perform  each  of  the  following 
steps  of  the  MDMP. 

Adequacy  of  Information 

General  Steps 

Adequate 

Inadequate 

Did  Not  Perform 

Receive  the  Mission 

[  3 

[  3 

[3 

Analyze  the  Mission 

[  3 

[  3 

[  3 

Development  of  COA 

[  3 

[  3 

[  3 

Analyze  the  COA  (War  Game) 

[  3 

[  3 

[  3 

Compare  the  COA 

[  3 

[  3 

[  3 

Approve  the  COA 

[  3 

[  3 

[  3 

Produce  the  Orders 

[  3 

[  3 

[  3 

At  times,  the  information  needed  to  perform  a  MDMP  step  is  available  in  the  system 
but  is  difficult  to  extract  from  the  system.  In  other  words,  the  information  is  available 
but  not  easily  accessible.  Please  rate  the  accessibility  of  the  information  needed  to 
perform  each  of  the  following  steps  of  the  MDMP. 

Accessibility  of  Information 


General  Steps  Adequate 

Receive  the  Mission  [  ] 

Analyze  the  Mission  [  ] 

Development  of  CO  A  [  ] 

Analyze  the  COA  (War  Game)  [  ] 

Compare  the  CO  A  .[  ] 

Approve  the  COA  [  ] 

Produce  the  Orders  [  ] 


Inadequate  Did  Not  Perform 


[  ] 
[  1 
[  1 
[  ] 
[  ] 
[  ] 
[  3 


[3 
[  3 
[  3 
[  3 
[  3 
[  3 
[3 
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8.  Please  rate  the  highest  level  of  workload  that  you  experienced  when  performing  each 
of  the  following  steps  of  the  MDMP. 


Your  Workload  Level 

Very 

Too 

Did  Not 

General  Steps 

High 

High 

Moderate 

Low 

Perform 

Receive  the  Mission 

[  ] 

[  1 

[  3 

[  3 

[  3 

Analyze  the  Mission 

[  ] 

[  ] 

[  3 

[  3 

[  3 

Development  of  COA 

[  ] 

[  ] 

[  3 

[  3 

[  3 

Analyze  the  COA  (War  Game) 

[  ] 

[  ] 

[  3 

[  3 

[  3 

Compare  the  COA 

[  1 

[  ] 

[  3 

[  3 

[  3 

Approve  the  COA 

[  ] 

[  ] 

[  3 

[  3 

[  3 

Produce  the  Orders 

[  ] 

[  ] 

[  3 

9.  For  today’s  mission,  were  errors  or  oversights  made  in  performing  any  of  the  MDMP 
steps? 

Errors  or  Oversights ? 


General  Steps 

Yes 

No 

Did  Not  Perform 

Receive  the  Mission 

[  3 

[  3 

[  3 

Analyze  the  Mission 

[  3 

[  3 

[] 

Development  of  COA 

[  3 

[  3 

[  3 

Analyze  the  COA  (War  Game) 

[  3 

[  3 

[  3 

Compare  the  COA 

[  3 

[  3 

[  3 

Approve  the  COA 

[  3 

[  3 

[  3 

Produce  the  Orders 

[  3 

[  3 

[  3 

If  you  answered  yes  to  any  of  the  above,  please  describe  below  each  error  or 
oversight  that  was  made  and  identify  the  MDMP  step  in  which  it  was  made. 


10.  For  today’s  mission,  did  lack  of  familiarity  with  the  operation  or  functional  capability 
of  equipment  in  the  digital  TOC  make  it  more  difficult  to  perform  any  of  the  MDMP 


steps? 

General  Steps 

More  Difficult? 
Yes  No 

Did  Not  Perform 

Receive  the  Mission 

[  3 

[  3 

[  3 

Analyze  the  Mission 

[3 

[3 

[  3 

Development  of  COA 

[  3 

[  3 

[  3 

Analyze  the  COA  (War  Game) 

[  3 

[  3 

[  3 

Compare  the  COA 

[  3 

[  3 

[  3 

Approve  the  COA 

[  3 

[  3 

[  3 

Produce  the  Orders 

[  3 

[  3 

[  3 

_  „  .  4  4  4 

If  you  answered  yes  to  any  of  the  above,  identify  the  equipment  for  which  you  lacked 
adequate  familiarity  and  the  MDMP  steps  that  were  influenced.  — - _ — - 
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1 1 .  For  today’s  mission,  were  any  of  the  TTPs  inadequate  for  performing  the  MDMP 
steps  in  a  digital  TOC? 

TTPs  Inadequate? 

General  Steps  Yes  No  Did  Not  Perform 

Receive  the  Mission  [  ]  [  ]  [  ] 

Analyze  the  Mission  [  ]  [  ]  [  ] 

Development  of  COA  [  ]  [  ]  [  ] 

Analyze  the  COA  (War  Game)  [  ]  [  ]  [  ] 

Compare  the  COA  [  ]  [  ]  [  ] 

Approve  the  CO  A  [  ]  [  ]  [  ] 

Produce  the  Orders  [  ]  [  ]  [  ] 

If  you  answered  yes  to  any  of  the  above,  identify  the  additional  or  modified  TTPs  that 
are  needed  and  the  MDMP  steps  that  would  be  improved.  _ _ 


12.  For  today’s  mission,  is  more  automation  needed  to  perform  adequately  the  MDMP 
steps  in  a  digital  TOC?  Check  “yes”  if  more  automation  is  needed  to  perform  the  step 
with  sufficient  accuracy,  comprehensiveness,  speed,  or  some  combination  of  these. 

Automation  Needed? 

General  Steps  Yes  No  Did  Not  Perform 

Receive  the  Mission  [  ]  [  ]  [  ] 

Analyze  the  Mission  [  ]  [  ]  [  ] 

Development  of  COA  [  ]  [  ]  [  ] 

Analyze  the  COA  (War  Game)  [  ]  [  ]  [  ] 

Compare  the  COA  [  ]  [  ]  [  ] 

Approve  the  CO  A  [  ]  [  ]  [  ] 

Produce  the  Orders  [  ]  [  ]  [  ] 

If  you  answered  yes  to  any  of  the  above,  identify  the  additional  automation  that  is 
needed  and  the  MDMP  steps  that  would  be  improved.  - - - 


13.  In  your  view,  is  there  a  need  to  add  a  staff  member  to  the  battalion  whose  sole 
responsibility  is  to  manage  the  flow  of  information  in  a  digital  TOC  during  mission 
planning. 

[  ]  No 

[  ]  Yes  . 

If  yes,  please  describe  as  specifically  as  you  can  the  duties  and  responsibilities  of  an 
information  manager.  _ _ _ _ _ _ 


14.  Please  rate  the  adequacy  of  vour  prior  training  for  performing  each  of  the  following 
MDMP  steps  in  a  digital  TOC. 

Adequacy  of  Training 

General  Steps 
Receive  the  Mission 
Analyze  the  Mission 
Development  of  CO  A 
Analyze  the  COA  (W ar  Game) 

Compare  the  COA 
Approve  the  CO  A 

If  you  checked  inadequate  for  any  of  the  MDMP  steps,  describe  the  additional 
training  that  you  believe  you  need. 


Adequate 

Inadequate 

Did  Not, 

[  ] 

[  ] 

[] 

[  ] 

[  ] 

[] 

[  1 

[] 

[  ] 

[  ] 

[  ] 

[] 

[  1 

[  ] 

[] 

[  ] 

[  1 

[] 

Your  answers  to  the  above  questions  are  extremely  valuable;  your  inputs  are  greatly 
appreciated  by  all  members  of  the  research  team.  If  you  have  other  insights  that  bear 
on  these  questions  (now  or  in  the  future),  please  contact  Dr.  William  Howse  (334) 
255-3686. _ _ _ 
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STEPS  IN  THE  MILITARY  DECISION  MAKING  PROCESS 
(By  Battle  Staff  Member) 

STEPS  IN  MILITARY  DECISION  MAKING  PROCESS 

1.  RECEIVE  THE  MISSION 


1.  Issue  a  WO  to  staff  personnel  _ 


2.  Determine  initial  timelines _ ___ _ 

3.  Issue  initial  commander’s  guidance _ , _ 


Receive  initial  commander’s  guidance _ _ 


4.  Issue  a  WO  to  subordinates  and  support  units _ 


STEPS  IN  MILITARY  DECISION  MAKING  PROCESS 
~~ — _  2.  ANALYZE  THE  MISSION 


1 .  Review  the  higher  headquarters  order 


Determine  staff  priorities/stds  &  coordinate  mission  analysis 
Review  known  enemy  and  friendly  situations _ 


2.  Conduct  initial  fPB  _ 


3.  Determine  specified,  implied,  and  essential  tasks _ 

4.  Determine  the  area  of  interest  (and  intent) _ 

5.  Review  available  assets  _ _ 


6.  Determine  constraints  _ _ 


Determine  and  maintain  the  status  of  personnel _ 

Determine  and  maintain  the  logistic  status _ 


Determine  FS  asset  allocation  status  _ 


Determine  air  defense  warning  status  _ 

7.  Identify  critical  facts  and  assumptions _ 

8.  Conduct  risk  assessment _ ___ _ 

9 .  Determine  initial  commander’ s  critical  information  reqts. 
Forecast  and  examine  personnel  reqts.  previously  identified 
Identify  areas  for  which  the  commander  is  responsible 


Compare  higher  hqs.  order  FA  data  to  AFATDS  data _ 

Determine  status  of  fire  units  that  are  providing  protection 

Identify  friendly  vulnerabilities  and  limitations _ 

Determine  capabilities  of  available  FA  assets  and  ammo 

10.  Determine  the  initial  reconnaissance  plan  _ 


Coordinate  external  logistics  support  _ 

Refine  timelines  _ _ _ 


Determine  liaison  requirements _ 


Identify  users  and  uses  of  airspace _ _ 

Identify  A2C2  treasures  imposed  by  higher  hqs. _ 

1 1 .  Plan  use  of  available  time  _ _ 


12.  Write  the  restated  mission 


13.  Conduct  a  mission  analysis  briefin 
Receive  mission  analysis  briefing 


14.  Approve  the  restated  mission _ _ _ 


15.  Develop  the  commander’s  intent _ _ _ 

Receive  commander’s  intent  and  planning  guidance _ 

16.  Issue  the  commander’s  guidance  _ 


17.  Issue  a  WO  to  subordinate  and  supporting  units _ 

1 8.  Review  facts  and  assumptions _ _ _ 


STEPS  IN  MILITARY  DECISION  MAKING  PROCESS 

3.  DEVELOPMENT  OF  COA  _ 

Conduct  direct  coordination  with  supported  ground  cdrs. 

1 .  Analyze  relative  combat  power _ 

Define  the  most  probable  and  most  dangerous  enemy  COAs 

2.  Generate  options _ _ _ _ 

3.  Array  initial  forces _ 

4.  Develop  a  scheme  of  maneuver _ _ 

Describe  the  effects  of  the  AO  and  enemy  situation  on  COA 

5.  Determine  C2  means _ _ _ 

Conduct  COA  briefing  _ 

Receive  COA  briefing _ _ _ 

6.  Prepare  COA  statements  and  sketches _ 


STEPS  IN  MILITARY  DECISION  MAKING  PROCESS 

4.  ANALYZE  THE  COA  (WAR  GAME)  ~ 

L  Gather  the  tools _ . _ _ 

2.  List  all  friendly  forces  _ _ _ _ 

3.  List  assumptions _ 

4.  List  known  critical  events  and  decision  points _ 

5.  Determine  evaluation  criteria _ 

6.  Select  the  war-game  method _ 

7.  Select  a  method  to  record  and  display  results _ 

8.  War  game  the  battle  and  assess  the  results _ 

Refine  COA  _ 


Identify  key  or  decisive  terrain  and  determine  how  to  use  it 

Determine  task  organization _ 

Identify  tasks  the  unit  must  retain  and  tasks  to  be  assigned 

Develop  a  synchronization  matrix _ 

Develop  decision  support  template _ 

Estimate  the  duration  of  each  critical  event  operation _ 

Identify  location  and  commitment  of  the  reserve _ 

Identify  the  location  of  the  cdr.  and  unit  CPs _ 

Identify  additional  critical  events  _ _ 

Determine  requirements  for  deception  and  surprise _ 

Determine  C2  requirements _ 

Determine  timing  of  force  concentration  and  attack _ 

Determine  movement  times  and  tables _ 

Develop  SEAD  plan _ „ 

Integrate  the  targeting  process _ 

Identify  risks,  determine  acceptable  risks,  &  risk  avoidance 
Project  the  percentage  of  total  enemy  forces  defeated 
Identity  likely  times  and  areas  for  enemy  use  of  WMD 
Refine  CCIR  and  incorporate  into  the  R&S  plan/graphics 
Identify  and/or  confirm  the  locations  of  decision  points 
Develop  the  intelligence  collection  and  dissemination  plan 

Integrate  the  targeting  process  _ 

Develop  scheme  of  fires _ _ _ 

Develop  execution  matrix _ 

Develop  target  template/target  list  worksheet _ 

Develop  fire  support  overlay _ 

Allocate  combat,  Cs,  and  CSS  assets  to  subordinate  cdrs. 

Develop  AD  plans/graphics _ ___ _ 

Develop  CSS  plans/graphics _ _ _ 
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STEPS  IN  MILITARY  DECISION  MAKING  PROCESS 

4.  ANALYZE  THE  COA  (WAR  GAME)  (continued) 

Identify  additional  requirements  for  CS 

Develop  CSS  plans/graphics 

Identify/analyze/evaluate  strengths  and  weaknesses  of  COA 

Conduct  war  game  briefing 

Receive  war  game  briefing  _ 

Assess  and  manage  risk 

STEPS  IN  MILITARY  DECISION  MAKING  PROCESS 

5.  COMPARE  THE  COA 

1  Analyze  and  evaluate  the  advantages  and  disadvantages 

Determine  and  consolidate  all  airspace  uses/users/conflicts 

2.  Present  each  staff  section’s  findings 

3  Compare  COA  utilizing  previously  defined  eval.  criteria 

Conduct  COA  decision  brief 

Receive  COA  decision  briefing 

4.  Select  recommended  COA 

STEPS  IN  MILITARY  DECISION  MAKING  PROCESS 

6.  APPROVE  THE  COA 

1  Conduct  commander’s  decision  briefing 

2.  Approve  COA 

3.  Issue  commander’s  guidance 

Repeat  portions  of  steps  3-5,  as  necessary 

4  Issue  WO  to  subordinate  and  supporting  units 
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APPENDIX  C 


ll.  Issue  a  WO  to  staff  personnel  _ I  J - 1 - 1 - 1— - 1 - 1 - \- 


2.  Determine  initial  timelines _  _ _ 


3.  Issue  initial  commander's  guidance _ _ 


Receive  initial  commander’s  guidance _ _ _ _ _ _ — — — - - — 

4.  Issue  a  WO  to  subordinates  and  support  units  _ l__J - - - L — — -l - - 

3 .  Check  the  support  staff  members  who  participated  in  Step  1 . 

[  ]  S-2  NCO  (UAV  Operator)  [  ]  S-3  NCO  (MCS/P  Operator) 

[  ]  S-2  NCO  (ASAS  Operator)  [  ]  S-3  NCO  (Other) 

[  ]  S-2  NCO  (UAV  Operator)  [  ]AS-3 

r  1 AS-2  (JSTARS  Operator)  [  ]  ADO  NCO  (AMD WS  Operator) 

[  ]AFATDS  Operator  [  ]  ADO  NCO  (FAADEO  Operator) 

4.  Identify  the  problems  observed  in  performing  Step  1  and  the  staff  members)  who 
encountered  the  problems. 

A.  Important  information  not  available - — - — 


B.  Important  information  not  easily  accessible . 


C.  Excessive  workload 


D.  Lack  of  knowledge  about  functional  capabilities  of  digital  equipment 


E.  Lack  of  skill  in  operating  digital  equipment . 


STEP  2.  ANALYZE  THE  MISSION 


5.  Timing:  Step  2  started _  Step  2  completed 


6.  Check  the  battle  staff  members  who  participated  in  each  Step  2  task. 


CDR 

XO 

S-l 

S-2 

S-3 

S-4 

FSO 

ADO 

1  Review  the  higher  headquarters  order 

Determine  staff  priorities/stds  &  coordinate  mission  analysis 

Review  known  enemy  and  friendly  situations 

2  Conduct  initial  IPB 

3  Determine  specified,  implied,  and  essential  tasks 

4  Determine  the  area  of  interest  (and  intent) 

5  Review  available  assets 

6  Determine  constraints 

Determine  constraints 

; .  • .  S' 

Determine  and  maintain  the  status  of  personnel 

7  Identify  critical  facts  and  assumptions 

8.  Conduct  risk  assessment 

9  Determine  initial  commander’ s  critical  information  reqts. 

Forecast  and  examine  personnel  reqts  previously  identified 

Identify  areas  for  which  the  commander  is  responsible 

Compare  higher  hqs  order  FA  data  to  AFATDS  data 

Determine  status  of  fire  units  that  are  providing  protection 

Identify  friendly  vulnerabilities  and  limitations 

Determine  capabilities  of  available  FA  assets  and  ammo 

10  Determine  the  initial  reconnaissance  plan 

f!nnniinate  external  logistics  support 

Refine  timelines  , 

Determine  liaison  requirements 

Identify  users  and  uses  of  airspace 

Identify  A2C2  measures  imposed  by  higher  hqs,  et.al. 

1 1  Plan  use  of  available  time 

12  Write  the  restated  mission 

13  Conduct  a  mission  analysis  briefing 

Receive  mission  analysis  briefing 

14  Approve,  the  restated  mission 

ppvplnp  the  commander's  intent 

Rpcoive  commander's  intent  and  planning  guidance 

... 

17  Issue  a  WO  to  subordinate  and  supporting  units 

1 8 .  Review  facts  and  assumptions 

7  Check  the  support  staff  members  who  participated  in  Step  2. 

^  p  ^  n  a  -v  -r  y—1  /—v  /I 


[  ]  S-2  NCO  (UAV  Operator) 

[  ]  S-2  NCO  (ASAS  Operator) 
[  ]  S-2  NCO  (UAV  Operator) 

[  ]  AS-2  (JSTARS  Operator) 

[  ]  AFATDS  Operator 


]  S-3  NCO  (MCS/P  Operator) 

]  S-3  NCO  (Other) 

]  A  S-3 

]  ADO  NCO  (AMDWS  Operator) 
]  ADO  NCO  (FAADEO  Operator) 
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8.  Identify  the  problems  observed  in  performing  Step  2  and  the  staff  member(s)  who 
encountered  the  problems. 

A.  Important  information  not  available  _ _ _ 


B.  Important  information  not  easily  accessible 


C.  Excessive  workload 


D.  Lack  of  knowledge  about  functional  capabilities  of  digital  equipment 


E.  Lack  of  skill  in  operating  digital  equipment 


F.  Inadequate  TTPs 


G.  Lack  of  training/experience  on  the  MDMP 


STEP  3.  DEVELOPMENT  OF  COA 

9.  Timing:  Step  3  started _  Step  3  completed 


10.  Check  the  battle  staff  members  who  participated  in  each  Step  3  task. 


CDR 

xo 

S-l 

S-2 

S-3 

S-4 

FSO 

ADO 

Conduct  direct  coordination  with  supported  ground  cdrs. 

1.  Analyze  relative  combat  power 

Define  the  most  probable  and  most  dangerous  enemy  COAs 

2.  Generate  options 

3.  Array  initial  forces 

4.  Develop  a  scheme  of  maneuver 

Describe  the  effects  of  the  AO  and  enemy  situation  on  COA 

5.  Determine  C2  means 

Conduct  COA  briefing 

Receive  COA  briefing 

6.  Prepare  COA  statements  and  sketches 

1 1 .  Check  the  support  staff  members  who  participated  in  Step  3. 

[  ]  S-2  NCO  (UAV  Operator)  [  ]  S-3  NCO  (MCS/P  Operator) 

[  ]  S-2  NCO  (ASAS  Operator)  [  ]  S-3  NCO  (Other) 

[  ]  S-2  NCO  (UAV  Operator)  [  ]AS-3 

[  ]  AS-2  (JSTARS  Operator)  (  ]  ADO  NCO  (AMDWS  Operator) 

[  ]  AFATDS  Operator  [  ]  ADO  NCO  (FAADEO  Operator) 

12.  Identify  the  problems  observed  in  performing  Step  3  and  the  staff  member(s)  who 
encountered  the  problems. 

A.  Important  information  not  available _ _ _ . 


B.  Important  information  not  easily  accessible 


C.  Excessive  workload 


D.  Lack  of  knowledge  about  functional  capabilities  of  digital  equipment 


E.  Lack  of  skill  in  operating  digital  equipment 


F.  Inadequate  TTPs 
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G.  Lack  of  training/experience  on  the  MDMP 


STEP  4.  ANALYZE  THE  COA  (WAR  GAME) 


13.  Timing:  Step  4  started _  Step  4  completed 


14.  Check  the  battle  staff  members  who  participated  in  each  Step  4  task. 


_ 4.  ANALYZE  THE  COA  (WAR  GAME) 

1.  Gather  the  tools _ 

2.  List  all  friendly  forces _ 

3.  List  assumptions _ 

4.  List  known  critical  events  and  decision  points _ 

5.  Determine  evaluation  criteria _ _ 

6.  Select  the  war-game  method _ 

7.  Select  a  method  to  record  and  display  results _ 

8.  War  game  the  battle  and  assess  the  results _ 

Refine  COA _ _ _ 

Identify  key  or  decisive  terrain  and  determine  how  to  use  it 

Determine  task  organization _ 

Identify  tasks  the  unit  must  retain  and  tasks  to  be  assigned 

Develop  a  synchronization  matrix _ 

Develop  decision  support  template _ 

Estimate  the  duration  of  each  critical  event  operation 

Identify  location  and  commitment  of  the  reserve _ 

Identify  the  location  of  the  commander  and  unit  CPs _ 

Identify  additional  critical  events _ 

Determine  requirements  for  deception  and  surprise _ 

Determine  C2  requirements _ 

Determine  timing  of  force  concentration  and  attack _ 

Determine  movement  times  and  tables _ 


CDR 


XO 


S-l 


S-2 


S-3 


S-4 


FSO 


ADO 


Develop  SEAD  plan _ 

Integrate  the  targeting  process _ 

Identify  risks,  determine  acceptable  risks,  &  risk  avoidance 

Pro  ject  the  percentage  of  total  enemy  forces  defeated _ 

Identify  likely  times  and  areas  for  enemy  use  of  WMD 
Refine  CCIR  and  incorporate  into  the  R&S  plan/graphics 
Identify  and/or  confirm  the  locations  of  decision  points 
Develop  the  intelligence  collection  and  dissemination  plan 

Integrate  the  targeting  process _ 

Develop  Scheme  of  fires _ 

Develop  execution  matrix _ 

Develop  target  template/target  list  worksheet  _ 

Develop  fire  support  overlay _ 

Allocate  combat,  Cs,  and  CSS  assets  to  subordinate  cdrs. 

Develop  AD  plans/graphics _ 

Develop  CSS  plans/graphics _ 

Identify  additional  requirements  for  CS _ _ 

Develop  CSS  plans/graphics  _ 

Identify/analyze/evaluate  strengths  and  weaknesses  of  COA 

Conduct  war  game  briefing _ 

Receive  war  game  briefing  _ 

Assess  and  manage  risk  _ 
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15.  Check  the  support  staff  members  who  participated  in  Step  4. 

[  ]  S-2NCO(UAV  Operator)  [  ]  S-3  NCO  (MCS/P  Operator) 

[  ]S-2  NCO  (ASAS  Operator)  [  ]  S-3  NCO  (Other) 

[  ]S-2  NCO  (UAV  Operator)  [  ]AS-3 

[  ]  AS-2  (JSTARS  Operator)  [  ]  ADO  NCO  (AMDWS  Operator) 

[  ]  AFATDS  Operator  [  ]  ADO  NCO  (FAADEO  Operator) 

16.  Identify  the  problems  observed  in  performing  Step  4  and  the  staff  member(s)  who 
encountered  the  problems. 

A.  Important  information  not  available _ _ _ _ — 


B.  Important  information  not  easily  accessible 


C.  Excessive  workload 


D.  Lack  of  knowledge  about  functional  capabilities  of  digital  equipment 


E.  Lack  of  skill  in  operating  digital  equipment 


F.  Inadequate  TTPs 


G.  Lack  of  training/experience  on  the  MDMP 


H.  Other 


STEP  5.  COMPARE  THE  COA 

17.  Timing:  Step  5  started _  Step  5  completed - 

18.  Check  the  battle  staff  members  who  participated  in  each  Step  5  task. 


CDR 

XO 

S-l 

S-2 

S-3 

S-4 

FSO 

ADO 

1  Analvze  and  evaluate  the  advantages  and  disadvantages 

Determine  and  consolidate  all  airspace  uses/users/conflicts 

2  Present  each  staff  section’ s  findings 

1  Compare  COA  utilizing  previously  defined  evaluation  criteria 

Conduct  COA  decision  brief 

Receive  COA  decision  briefing 

4.  Select  recommended  COA 

19  Check  the  support  staff  members  who  participated  in  Step  5. 

[  ]S:2NCO(UAV  Operator)  [  ]  S-3  NCO  (MCS/P  Operator) 

[  ]  S-2  NCO  (ASAS  Operator)  [  ]  S-3  NCO  (Other) 

[  1  S-2  NCO  (UAV  Operator)  [  ]AS-3 

[  ]  AS-2  (JSTARS  Operator)  [  ]  ADO  NCO  (AMD WS  Operator) 

[  ]  AFATDS  Operator  [  ]  ADO  NCO  (FAADEO  Operator) 

20.  Identify  the  problems  observed  in  performing  Step  5  and  the  staff  member(s)  who 
encountered  the  problems. 

A.  Important  information  not  available  - - - - — 


B.  Important  information  not  easily  accessible 


C.  Excessive  workload 


D.  Lack  of  knowledge  about  functional  capabilities  of  digital  equipment 


E.  Lack  of  skill  in  operating  digital  equipment 


F.  Inadequate  TTPs 


G.  Lack  of  training/experience  on  the  MDMP 


STEP  6.  APPROVE  THE  COA 


21.  Timing:  Step  6  started _  Step  6  completed _ 

22.  Check  the  battle  staff  members  who  participated  in  each  Step  6  task. 


CDR 

XO 

S-l 

S-2 

S-3 

S-4 

FSO 

ADO 

1 .  Conduct  commander’s  decision  brief 

2.  Approve  COA 

3.  Issue  commander’s  guidance 

Repeat  portions  of  steps  3-5,  as  necessary 

4.  Issue  WO  to  subordinate  and  supporting  units 

*■ 

23.  Check  the  support  staff  members  who  participated  in  Step  6. 

[  ]  S-2  NCO  (UAV  Operator)  [  ]  S-3  NCO  (MCS/P  Operator) 

[  ]  S-2  NCO  (ASAS  Operator)  [  ]  S-3  NCO  (Other) 

[  ]  S-2  NCO  (UAV  Operator)  [  ]AS-3 

[  ]  AS-2  (JSTARS  Operator)  [  ]  ADO  NCO  (AMDWS  Operator) 

[  ]  AFATDS  Operator  [  j  ADO  NCO  (FAADEO  Operator) 

24.  Identify  the  problems  observed  in  performing  Step  6  and  the  staff  member(s)  who 
encountered  the  problems. 

A.  Important  information  not  available _ _ _ 


B.  Important  information  not  easily  accessible 


C.  Excessive  workload 


D.  Lack  of  knowledge  about  functional  capabilities  of  digital  equipment 


E.  Lack  of  skill  in  operating  digital  equipment 


F.  Inadequate  TTPs 


G.  Lack  of  training/experience  on  the  MDMP 


C-ll 
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OVERALL  OBSERVATIONS 


25.  Did  you  observe  any  mission  planning  errors  or  oversights  that  were  important 
enough  to  reduce  the  likelihood  of  a  fully  successful  mission  outcome? 

[  ]No 
[  ]  Yes 

If  yes,  describe  each  error  or  oversight  that  you  observed  and  the  staff  member(s) 
who  made  the  error  or  oversight. _ _ _ 


26.  Did  you  observe  any  evidence  that  increased  automation  is  needed  to  enable  staff 
members  to  perform  some  tasks  with  sufficient  accuracy,  comprehensiveness,  or 
timeliness? 

[  ]No 
[  ]Yes 

If  yes,  describe  the  task(s)  for  which  you  think  increased  automation  is  needed. _ 


27.  Did  you  observe  any  evidence  that  there  is  a  need  to  add  a  staff  member  to  the 

battalion  battle  staff  whose  sole  responsibility  is  to  manage  the  flow  of  information  in 
a  digital  TOC  during  mission  planning? 

[  ]No 
[  ]  Yes 

If  yes,  explain  the  reasons  why  you  think  an  information  manager  is  needed  during 
mission  planning.  _ _ _ _ : - 


28.  Did  you  observe  any  evidence  that  one  or  more  member  of  the  battle  staff  lacked  the 
prior  training  that  was  needed  to  perform  his/her  tasks  during  mission  planning? 

[  ]No 
[  ]  Yes 

If  yes,  identify  the  staff  member  whose  prior  training  was  inadequate  and  the  task(s) 
that  were  influenced  by  the  training  deficiency.  _ _ 
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Mission  Execution  Observations 
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COMMUNICATION  EVENTS  _ _  _  r _ 7 _  OTHER  PROS 


Observer’s  Name 
Date 

Mission  Number 
Observation  Station 


Time 

From 


DEFINITION  OF  TERMS  FOR 
MISSION  EXECUTION  OBSERVATION  FORM 


ITEM  COMMENTS 

None 

None 

The  missions  will  be  numbered  consecutively  beginning  with  the  first  mission  following 
the  rehearsal  or  practice  missions  (if  any). 

Station  1  will  observe 

-  S-2 

-  JSTARS  Operator  (AS-2) 

-  UAV  Operator  (S-2  NCO) 

Station  2  will  observe 

-  -ASAS  Operator  (S-2  NCO) 

-  Battle  Captain 

-  Assistant  Battle  Captain 

-  MCS/P  Operator  (S-3  NCO) 

Station  3  will  observe 

-  S-3  (when  present  in  TOC) 

-  AS-3  (?) 

-  Printer  Operator  (S-3  NCO) 

-  FAADEO  Operator  (ADO) 

-  AMDWS  Operator  (ADO  NCO) 

Station  4  will  observe 

-  Battalion  Commander  (when  present) 

Deputy  Commander 

-  FSO 

-  AFATDS  Operator 

Station  5  will  serve  as  backup  for  other  four  stations 

Enter  the  time  elapsed  since  the  mission  started.  Record  the  time  of  all  communication 
events  and  all  “other  problems/events”  that  are  discussed  below. 

Enter  the  identification  code  of  the  battle  staff  member  who  initiated  the  communication 
event.  Use  the  following  codes. 

C  =  Commander 
X  =  XO  (Deputy  Commander) 

B  =  Battle  Captain 

AB  =  Assistant  Battle  Captain 

2  =  S-2 

J  =  JSTARS  Operator  (Assistant  S-2) 

U  =  UAV  Operator  (S-2  NCO) 

A  =  ASAS  Operator  (S-2  NCO) 

3  =  S-3 

M  =  MCS/P 
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To 

Request 

Respond 

Push 

Situation  Update 

Coordinate 

Message  Content 
Workload 
Sit.  Awareness 

Decision  Pt1 
Decision  E2 

Skill  Def 3 
Other 

1  Pt.  =  point 

2  E  =  error 

3  Def  =  deficiency 


P  =  Printer  Operator  (S-3  NCO) 

F  =  FSO 

AF  =  AFATDS  Operator 

FA  =  FAADEO  Operator  (ADO) 

AM  =  AMDWS  Operator  (ADO  NCO) 

Enter  the  identification  code  of  the  battle  staff  member  who  was  the  main  recipient  of  the 
information.  Use  the  same  codes  as  listed  above. 

Check  the  corresponding  cell  if  the  communication  event  was  a  request  for  information. 
That  is,  the  communication  event  consists  of  one  staff  member  requesting  information 
from  another  staff  member. 

Check  the  corresponding  cell  if  the  communication  event  was  a  response  to  a  prior 
request  for  information. 

Check  the  corresponding  cell  if  one  staff  member  conveys  information  to  another  staff 
member  without  being  requested  (explicitly)  to  do  so. 

Check  the  corresponding  cell  if  the  communication  event  is  a  general  situation  update 
provided  by  one  staff  member  for  the  benefit  of  all  or  most  of  the  other  staff  members. 

Check  the  corresponding  cell  if  the  purpose  of  the  communication  event  is  to  coordinate 
the  activities  of  the  battle  staff  members  or  the  activities  external  persons/units. 

As  briefly  as  possible,  describe  the  content  of  the  message. 

Check  the  corresponding  cell  if  a  staff  member’s  workload  is  considered  excessive. 

Check  the  corresponding  cell  if  a  staff  member  clearly  lacks  a  sufficient  level  of  situation 
awareness.  Interpretation  of  a  staff  member’s  verbal  responses  and  overt  actions  is  the 
only  way  to  assess  situation  awareness  through  casual  observation. 

Check  the  corresponding  cell  if  the  situation  requires  a  staff  member  to  make  a  decision. 

Check  the  corresponding  cell  if  a  decision  made  by  a  staff  member  has  an  adverse  effect 
on  the  mission  outcome.  Decision  error  may  not  apparent  at  the  time  the  decision  is 
made  but  become  obvious  as  the  mission  proceeds. 

Check  the  corresponding  cell  if  you  observe  evidence  that  a  staff  member  lacks  the  skills 
needed  to  perform  his/her  assigned  job. 

Check  the  corresponding  cell  if  you  observe  other  events  that  may  be  important  to  the 
objectives  of  the  study.  These  events  may  simply  be  events  that  needed  to  be  discussed 
and/or  clarified  during  the  AAR. 
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Staff  Member/Notes 


Page 


In  the  space  provided  in  this  column,  record  the  staff  member  who  experienced  the 
problem/event  and  make  the  notes  you  need  to  jog  your  memory  about  the  event  that 
occurred. 

In  the  space  provided,  number  sequentially  all  of  the  pages  used  during  a  mission 
execution  exercise. 
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APPENDIX  E 


Battle  Staff  Training  Effectiveness  Survey 

We  would  like  your  input  regarding  the  use  of  the  ATCCS  suite  and  virtual  simulations 

as  instructional  media  for  battle  staff  training.  Please  respond  to  the  items  below  by 

indicating  the  degree  to  which  you  agree  with  the  statement.  For  each  item,  mark  the  box 
that  corresponds  to  your  level  of  agreement.  Although  you  are  certainly  fatigued  by  this 
point  and  want  only  to  get  out  of  here,  please  take  a  moment  to  make  comments  or 
suggestions,  positive  or  negative,  at  the  end  of  the  form. 

Thank  you  for  all  your  effort.  The  work  you  have  done  this  week  will  help  the  Army  to 
be  better  prepared  for  war. - - - - - - 

1  I  feel  I  gained  substantial  benefit  from  the  exercise. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

2  I  know  more  now  than  I  did  before  the  exercise. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

3  The  ATCCS  exercise  was  interesting  and  enjoyable. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

4  The  exercise  exceeded  my  expectations. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 


5.  The  ATCCS  environment  was  well  designed. 


□ 

□ 

□ 

□  □ 

Strongly 

Mildly 

Neutral 

Mildly  Strongly 

Disagree 

Disagree 

Disagree  Disagree 

6.  Observers/Controllers  provided  beneficial  instruction. 

□ 

□ 

□ 

□  □ 

Strongly 

Mildly 

Neutral 

Mildly  Strongly 

Disagree 

Disagree 

Disagree  Disagree 

7  The  skills  required  in  the  ATCCS  exercise  are  important  during  field  exercises. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 
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8  The  feedback  I  received  during  this  exercise  was  very  important. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

9  The  ATCCS  suite  provides  significant  improvement  in  information  exchange. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

10  The  ATCCS  scenarios  well  simulated  actual  battlefield  engagements. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

111  intend  to  apply  what  I  learned  here  in  field  assignments. 

□  □□□•□ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

12  The  ATCCS  exercise  was  a  useful  learning  experience. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

13  1  believe  that  this  experience  will  help  me  perform  better  as  a  staff  officer. 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

14  I  intend  to  use  what  I  learned  during  this  exercise  in  future  assignments. 

□  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

15  1  was  well  prepared  for  this  training  exercise. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

16  This  training  helped  improve  my  leadership  skills. 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

17  The  digitized  TOC  enhances  battalion  staff  capabilities. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

18  The  exchange  of  information  through  digitized  communication  was  easy. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 
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19  The  observers  provided  good  analyses  of  the  battalion  staff  performance. 

'□□□□□ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

20  Planning  and  executing  missions  is  easier  in  a  digitized  TOC. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

21  The  ATCCS  lasted  the  right  amount  of  time  to  be  useful. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

22  I  acquired  new  skills  from  the  ATCCS  exercise. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 


23  I  would  like  to  participate  in  other  similar  exercises. 

□  □  □  □  □ 

Strongly  Mildly  Neutral  Mildly  Strongly 

Disagree  Disagree  Disagree  Disagree 

24  I  learned  a  great  deal  during  the  ATCCS  exercise. 


Strongly 

Mildly 

Neutral 

Mildly 

Strongly 

Disagree 

Disagree 

Disagree 

Disagree 

25.  This  training  will  have 

a  positive  effect 

on  my  future  performance. 

□ 

□ 

□ 

□ 

□ 

Strongly 

Mildly 

Neutral 

Mfldly 

Strongly 

Disagree 

Disagree 

Disagree 

Disagree 

26.  This  was  a  good  learning  experience  for  me. 

□ ; 

□ 

□ 

□ 

□ 

Strongly 

Mildly 

Neutral 

Mildly 

Strongly 

Disagree 

Disagree 

Disagree 

Disagree 

27.  Mission  planning  is  faster  in  the  digitized  TOC. 


□ 

□ 

□ 

□ 

□ 

Strongly 

Mildly 

Neutral 

Mildly 

Strongly 

Disagree 

Disagree 

Disagree 

Disagree 

28. 1  enjoyed  myself  during  this 1 

exercise. 

□ 

□ 

□ 

□ 

□ 

Strongly 

Mildly 

Neutral 

Mildly 

Strongly 

Disagree 

Disagree 

Disagree 

Disagree 

29. 1  have  a  better  level  of  ability  as  a  staff  officer  because  of  this  exercise. 


□ 

□ 

□ 

□ 

□ 

Strongly 

Mildly 

Neutral 

Mildly 

Strongly 

Disagree 

Disagree 

Disagree 

Disagree 
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30  The  AAR  process  was  a  useful  learning  tool. 

□  □  □  □  □ 


Strongly  Mildly  Neutral  Mildly  Strongly 
Disagree  Disagree  Disagree  Disagree 


3 1 .  Your  comments  and  suggestions  regarding  the  exercise: 


ORIGINAL  SCALES  FOR  TRAINING  EFFECTIVE  QUESTIONNAIRE 


Scale  A:  Learning 

•  I  learned  a  great  deal  during  the  ATCCS  exercise. 

•  I  know  more  now  than  I  did  before  the  exercise. 

•  This  was  a  good  learning  experience  for  me. 

•  I  acquired  new  skills  from  the  ATCCS  exercise. 

•  I  have  a  better  level  of  ability  as  a  staff  officer  because  of  this  exercise. 

Scale  B:  Exercise  conduct 

•  I  was  well  prepared  for  this  training  exercise. 

•  The  ATCCS  environment  was  well-designed. 

•  The  ATCCS  scenarios  well  simulated  actual  battlefield  engagements. 

•  The  skills  required  in  the  ATCCS  exercise  are  important  during  field  exercises. 

•  The  ATCCS  lasted  the  right  amount  of  time  to  be  useful. 

Scale  C:  Future  application 

•  I  intend  to  apply  what  I  learned  here  in  field  assignments. 

•  I  believe  that  this  experience  will  help  me  perform  better  as  a  staff  officer. 

•  This  training  will  have  a  positive  effect  on  my  future  performance. 

•  I  intend  to  use  what  I  learned  during  this  exercise  in  future  assignments. 

•  This  training  helped  improve  my  leadership  skills. 

Scale  D:  Digital  system  utility 

•  The  exchange  of  information  through  digitized  communication  was  easy. 

•  The  digitized  TOC  enhances  battalion  staff  capabilities. 

•  Planning  and  executing  missions  is  easier  in  a  digitized  TOC. 

•  The  ATCCS  suite  provides  significant  improvement  in  information  exchange. 

•  Mission  planning  is  faster  in  the  digitzed  TOC. 

Scale  E:  Instructional  feedback 

•  The  feedback  I  received  during  this  exercise  was  very  important. 

•  The  AAR  process  was  a  useful  learning  tool. 

•  The  observers  provided  good  analyses  of  the  battalion  staff  performance. 

•  Observer  /  Controllers  provided  beneficial  instruction. 

•  The  ATCCS  exercise  was  a  useful  leraning  experience. 

Scale  F:  Enjoyment 

I  enjoyed  myself  during  this  exercise. 

•  This  exercise  exceeded  my  expectations. 

• '  I  would  like  to  participate  in  other  similar  exercises. 

•  I  feel  I  gained  substantial  benefit  from  the  exercise. 

•  The  ATCCS  exercise  was  interesting  and  enjoyable. 
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APPENDIX  F 


Mission  Planning  Observations: 

Participants  Who  Performed  the  MDMP  Substeps 

Notes  on  The  Interpretation  of  the  Following  Data  Tables 

1 .  The  numbers  in  the  cells  refer  to  dates  on  which  mission  planning  was  observed  (2  = 
March  2,4  =  March  4,  5  =  March  5).  Mission  1  was  planned  on  March  2,  Mission  2 
was  planned  on  March  4,  and  Mission  3  was  planned  on  March  5. 

2.  A  numbers  in  a  cell  indicates  that  the  corresponding  battle  staff  member  participated 
in  the  corresponding  substep  on  the  dates  indicated.  The  cell  entry  “2,  5”  indicates 
that  the  corresponding  substep  was  performed  by  the  corresponding  batde  staff 
member  on  both  March  2  (Mission  1)  and  March  5  (Mission  3). 

3 .  A  shaded  cell  indicates  the  battle  staff  members  ordinarily  participate  in  the 
performance  of  the  corresponding  substep  (as  specified  in  FM  100-5). 

4.  The  substep  numbers  are  arbitrary. 


BATTLE  STAFF  MEMBER 

SUBSTEPS  IN  MDMP  (RECEIVE  THE  MISSION) 

CDR 

XO 

S-l 

S-2 

S-3 

S-4 

FSO 

ADO 

1. 

Issue  a  WO  to  staff  personnel 

2. 

Determine  initial  timelines 

5 

3. 

Issue  initial  commander’s  guidance 

2  r 

5 

■  ■ 

■ 

■■ 

3a. 

Receive  initial  commander’s  guidance 

2  f 

EM 
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Acronyms  and  Abbreviations 

First  Battalion  of  the  145th  Aviation  Regiment,  First  Aviation  Brigade 

4ID 

Fourth  Infantry  Division 

AAR 

After  Action  Review 

ABC 

Assistant  Battle  Captain 

ABCS 

Army  Battle  Command  System 

ABM 

Air  Battle  Management 

ADA 

Air  Defense  Artillery 

ADO 

Air  Defense  Officer 

AFATDS 

Advanced  Artillery  Tactical  Data  System 

AHB 

Attack  Helicopter  Battalion 

ALOC 

Administrative  and  Logistics  Operations  Center 

AMBL 

Air  Maneuver  Battle  Lab 

AMDW/S 

Air  and  Missile  Defense  Workstation 

AMPS 

Aviation  Mission  Planning  System 

ARI 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 

AS2 

Assistant  S2 

AS3 

Assistant  S3 

AS  AS 

All  Source  Analysis  System 

ASTi 

Advanced  Simulation  Technology,  Inc. 

ATACMS 

Army  Tactical  Missile  System 

ATCCS 

Army  Tactical  Command  and  Control  System 

G-l 


ATHS 


Automated  Target  Handover  System 
AVO  AC  Aviation  Officers  Advanced  Course 

AVTB  Army  Aviation  Test  Bed 

AWSC  Aviation  Warfighting  Simulation  Center 

BADD  Battlefield  Awareness  Data  Dissemination  system 

BC  Battle  Captain 

BFA  Battlefield  Functional  Area 

BPV  Battlefield  Planning  /  Visualization  system 

BRIL  Baseline  Resource  Items  List 

C4I  Command,  Control,  Communications,  Computers  and  Intelligence 

CAS  Close  Air  Support 

CGF  Computer  Generated  Forces 

CGS  Common  Ground  Station 

CO  Commanding  Officer 

COA  Course  of  Action 

CSAR  Combat  Search  and  Rescue 

i 

CSS  Combat  Service  Support 

CSSCS  Combat  Service  Support  Control  System 

CTIL  Commander's  Tracked  Items  List 

DCD  Directorate  of  Combat  Developments 

DIS  Distributed  Interactive  Simulation 
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DOTDS 

Directorate  of  Training,  Doctrine  and  Simulation 

DP 

Decision  Point 

DST 

Decision  Support  Template 

DTSS 

Digital  Topographical  Support  System 

EAC 

Echelons  Above  Corps 

EADSIM 

Extended  Air  Defense  Simulation 

EO 

Engagement  Operations 

EPLRS 

Enhanced  Position  Location  Reporting  System 

FAADC2 

Forward  Air  Defense  Command  and  Control  system 

FAADC2I 

Forward-Area  Air  Defense  Command,  Control  and  Intelligence 

FAADEO 

Forward- Area  Air  Defense  Engagement  Operations 

FAADFO 

Forward-Area  Air  Defense  Force  Operations 

FARP 

Forward  Area  Refueling  Point 

FBCB2 

Force  XXI  Battle  Command  Brigade  and  Below 

FLOT 

Forward  Line  of  Own  Troops 

FM 

Frequency  Modulation 

FO 

Force  Operations 

FRAGO 

Fragmentary  Orders 

FRED 

Fully  Reconfigurable  Experimental  Device 

FSO 

Fire  Support  Officer 

FTP 

File  Transfer  Protocol 

GCCS-A 

Global  Command  and  Control  System  -  Army 

GS 

General  Support 
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BEW 

Intelligence  and  Electronic  Warfare 

IMETS 

Integrated  Meteorological  System 

INC 

Integrated  Network  Controller 

IPB 

Intelligence  Preparation  of  the  Battlefield 

IWEDA 

Integrated  Weather  Effects  Decision  Aid 

JSTARS 

Joint  Surveillance  Target  Attack  Radar  System 

LAN 

Local  Area  Network 

LOS 

Line  of  Sight 

MCOO 

Modified  Combined  Obstacle  Overlay 

MCS 

Maneuver  Control  System 

MCS 

Maneuver  Control  System 

MDMP 

Military  Decision  Making  Process 

MLRS 

Multiple  Launched  Rocket  System 

ModSAF 

Modular  Semi- Automated  Forces 

MSE 

Mobile  Subscriber  Equipment 

MTI 

Moving  Target  Indicator 

NAI 

Named  Area  of  Interest 

NCO 

Noncommissioned  Officer 

NDM 

Naturalistic  Decision  Making 

NTDR 

Near  Term  Digital  Radio 

ODCST 

Office  of  the  Deputy  Chief  of  Staff  for  Training 

OPCON 

Operational  Control 

OPFOR 

Opposing  Force 
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OPORD 

Operational  Order 

PATRIOT 

Phased  Array  Tracking  Intercept  of  the  Target  missile 

PIU 

Protocol  Interface  Unit 

PMOS 

Primary  Military  Occupational  Specialty 

RWS 

Remote  Workstation 

SI 

Personnel  Officer 

S2 

Intelligence  Officer 

S3 

Operations  and  Training  Officer 

S4 

Supply  Officer 

SIMNET 

Simulation  Network 

SINCGARS 

Single  Channel  Ground  Air  Radio  System 

SME 

Subject  Matter  Expert 

SOP 

Standard  Operating  Procedure 

SSI 

Specialty  Skill  Indicator 

STAMIS 

Standard  Army  Management  Information  System 

TAFSIM 

Target  Acquisition  Fire  Support  Model 

TAI 

Target  Area  of  Interest 

TEL 

Threat  Transporter-Erector-Launcher  missile 

TO&E 

Table  of  Organization  and  Equipment 

TOC 

Tactical  Operations  Center 

TRADOC 

Training  and  Doctrine  Command 

TSIU 

Tactical  Simulation  Interface  Unit 

TSP 

Training  Support  Package 
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TIP 

Tactics,  Techniques  and  Procedures 

TUAV 

Tactical  Unmanned  Aerial  Vehicle 

UAV 

Unmanned  Aerial  Vehicle 

USAAVNC 

U.  S.  Army  Aviation  Center 

USMTF 

United  States  Message  Text  Format 

WFD 

Warfighting  Futures  Division,  DOTDS 

WAN 

Wide  Area  Network 

xo 

Executive  Officer 
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